Contact Web Status Presentation for Freehand Grasping in MR-based Robot-teaching

Presenting realistic grasping interaction with virtual objects in mixed reality (MR) is one of the important issues to promote the use of MR, such as in the robot-teaching domain. However, the intended interaction is difficult to achieve in MR due to the difficulty of depth perception and the lack of haptic feedback. To make intended grasping interactions in MR easier to achieve, we propose visual cues (contact web) that represent the contact state between the user’s hand and a MR-object. To evaluate the effect of the proposed method, we performed two experiments. The first experiment determines the grasp type and object to be used in the evaluation of the second experiment, and the second experiment measures the time taken to complete grasping tasks of the object. Both objective and subjective evaluations show that the proposed visual cues entailed a significant reduction in the time required to complete the task.