Oxymorons of haptics might be the new normal
Consumer virtual reality (VR) systems can immerse users in wonderous virtual worlds, rich in sights and sounds. However, how many times have you reached for an object in VR or augmented reality (AR) and, just as you were about to grab it, you experienced the bitter sensorial shock that the object did not exist in the real world. That break on presence could be about to end. Researchers have explored many compelling forms of haptic feedback ranging from exoskeletons to haptic controllers, but the mechanical complexity of the proposed systems increases their cost and reduces their reliability, limiting their acceptance outside the lab.
Now, researchers Mike Sinclair, Eyal Ofek, Mar Gonzalez-Franco and Christian Holz of Microsoft Research Redmond, in collaboration with research intern Jaeyeon Lee from KAIST, present a new kind of haptic controller called TOuch Rigid Controller – TORC – to be unveiled at the ACM CHI Conference on Human Factors in Computing Systems (CHI 2019) May 4-9 in Glasgow, United Kingdom.
TORC is a compact device that fits on the palm of your hand. It is composed of a rigid shell that has no visible moving parts, yet it can deliver rich haptic feedback and allows high level of dexterity and compliance perception when manipulating virtual objects (see Figure 1.)
Microsoft research podcast
What’s Your Story: Lex Story
Model maker and fabricator Lex Story helps bring research to life through prototyping. He discusses his take on failure; the encouragement and advice that has supported his pursuit of art and science; and the sabbatical that might inspire his next career move.
Figure 1: Using TORC to manipulate a virtual object with dexterity and compliance.
TORC is designed to support gripping virtual objects using the thumb and two fingers (see Figure 2.) The user can move a thumb freely on the surface of TORC parallel to the plane of the two fingers to rotate a virtual object. At the same time, rendered vibrations let the fingers feel the object’s properties such as texture and pseudo-compliance.
Figure 2: The TORC controller is designed as a hand-held device with no external moving parts.
In fact, the impetus for creating TORC was to be able to convey the haptic feeling of compliance of a silicone ball. As such, TORC will not only let you touch and grab objects, but, for example, it could also render an anti-stress ball that can help you deal with your virtual frustrations. When holding an elastic object, applying force on TORC squeezes the virtual object. Although the user’s physical fingers do not move, the combination of the visual stimuli accompanied with the rendered haptic sensation combines to form a compelling experience (see Figure 3.)
Figure 3: Applying force on the controller allows users to squeeze virtual objects. A simulated haptic sensation rendered on TORC completes the visual simulation. Compliance tests of TORC in VR showed it can be used to simulate a silicone anti-stress ball.
But TORC can do much more as it enables the natural ability of the human hand to have precise fingertip movements. This dexterity of finger movement allows for detailed and easy manipulation of virtual tools and objects in a way unparalleled by todays commercial VR or AR controllers. For example, if the user grabs a virtual key with TORC, they will be able to explore it, insert it in a keyhole and turn it easily to unlock a door (see Figure 4.)
Figure 4: Using TORC to insert a key on a keyhole. TORC allows precise manipulation of a held object’s rotation by sensing finger motion.
Finally, the force sensor in TORC can be used to acquire or release objects. By approaching a virtual object and causing the thumb to touch the 2D position sensor on TORC, the object will be grabbed. By lifting the thumb from the device, or sensing too little force between thumb and fingers to maintain the grasp, the object will be released (see Figure 5.)
Figure 5: Applying a no-pressure and lifting the thumb releases the grabbed object.
This new ability to squeeze virtual objects as well as manipulate them in-hand, without having to switch modes in a multifunctional way, could be considered an ultimate VR frontier in which users will finally be able to grab any object within VR and AR environments.
Embedded on other input devices
TORC’s minimal design enables simple and robust operation and it can be incorporated into a variety of controllers and everyday devices, from a stylus to a game controller (see Figure 6.) It enables sensing and rendering rich haptic manipulation in VR and AR. The TORC project team hopes that it will motivate the research and industrial community to pursue future work on the design of VR/AR controllers.
Figure 6: The small size and simplicity of TORC can fit to many form factors.
The neuroscience behind TORC
The TORC project team enabled natural haptic interactions, such as squeezing a virtual silicone ball, by translating multisensory integration theories to the world of hardware prototypes. TORC working fundamentals rely on the fact that humans are very susceptible to dynamic visuotactile stimulation. When provided in the right way the force applied by the user on our device will create equivalent wide-band Voice Coil Actuation (VCA). The real-time effect within virtual reality creates the necessary proprioceptive drifts that enable different compliances for virtual objects.
We look forward to seeing you at CHI 2019 in Glasgow as we demo TORC and hear your impressions.
Human-computer interaction