The notion of an “uncanny valley” is well known in robotics[1]. This is when increasing the realism of a robot’s human-like appearance can lead to feelings of unease—or even revulsion—as a robot becomes more and more (but never quite fully) human-like[2].
While this uncanny valley is related to sight, researchers wondered if an uncanny valley experience also exists for the sense of touch—also known as haptics.
Microsoft Research Blog
Microsoft at CHI 2024: Innovations in human-centered design
From immersive virtual experiences to interactive design tools, Microsoft Research is at the frontier of exploring how people engage with technology. Discover our latest breakthroughs in human-computer interaction research at CHI 2024.
In a paper recently published in Science Robotics (opens in new tab)[3], a group of Microsoft researchers put forward a new theory of haptics interaction that identifies an uncanny valley for the sense of touch. Their research suggests that when the haptics are not aligned with the rest of the sensory stimulation, it can hinder the experience. The researchers believe that this research will create a new school of thought among designers of touch-enabled interactions and devices.
Researchers Mar Gonzalez Franco (opens in new tab), Eyal Ofek (opens in new tab), Ken Hinckley (opens in new tab) and Christopher Berger (opens in new tab), a visiting researcher from California Institute of Technology, chose the platform of virtual reality (VR) as a testbed. VR is a great tool for revealing the strange and otherworldly foundation of our everyday sensory perceptions. We can use VR to produce a sensation of touch that feels like it originates in the completely empty space between one’s hands, while holding two controllers.
Each controller provides the user with synchronous vibrotactile stimuli of different amplitudes. In the virtual environment, we show the participant a wooden dowel that they grip with each hand. The dowel is only imaginary, of course; the hands are not physically linked and the dowel does not exist. We then show a small white marble that seems to knock on the dowel at different locations as we vary the vibratory sensations.
Figure 1: Top: A diagram showing the stimulation points for the spatialized touch generation. Middle: The participant holding the controllers. Bottom: First-person perspective view of their virtual hands and the stick. The amorphous cloud shown here reduces the uncanny valley effect.
What’s interesting is that not only is the dowel imaginary, but also that the apparent spatial location of the touch sensation of the marble will persist, even if the marble is invisible. This is because the brain follows the unity rule by which two synchronous events that are connected in time should also be connected in space, thus they have the same cause and origin. In this case, the two vibrations felt in the hands are perceived to be the result of a single touch that comes from the same location along the virtual dowel, no matter if the marble is visible or not.
In virtual environments, producing higher-fidelity haptic sensations of this sort come with the assumption that such improvements will always yield a more realistic and immersive experience. However, this research found that adding the spatialized touch on its own, without the visual counterpart of the marble, can greatly reduce the perceived realism of and immersion into the virtual environment. That is despite the fact that participants were able to locate the touch sensation equally well when the marble was invisible or visible.
The video shows the newly found uncanny valley, where an increased quality of haptics generates a drop in the subjective experience. However, this uncanny valley can be shifted or eliminated by subtly manipulating the experimental conditions. The researchers found that by rebalancing the ray of the sensorial experience, particularly by increasing the visual quality to provide a plausible source for the haptic experience, a fuller sense of immersion returns and increases the overall subjective experience.
Likewise, the sensation can be manipulated based on whether we are simply holding the virtual dowel—thus experiencing the knock of the marble as a passive participant—or actively moving the dowel up and down, thus becoming the agent of the sensation through our own actions, which again increases the sense of realism and immersion.
Understanding these effects more deeply helps us build better and more convincing virtual environments. The result is a key insight that lends a deeper understanding of how human perception really works and offers a boost to future endeavors that can truly increase the sensory realism of all of our would-be virtual dreams. The researchers believe that this demonstration of the newly found uncanny valley of haptics, and additional insights on how to overcome the haptic touch revulsion, are critical not only in VR controllers and robotic teleoperation, but also across all types of touch-enabled devices.
[1] Mori, M. Translated by MacDorman, K. F. and Kageki, Norri. The uncanny valley (opens in new tab). IEEE Robotics and Automation. 2012.
[2] Chattopadhyay, D., & MacDorman, K. F. Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley. Journal of Vision, 16. 2016.
[3] Berger, C.C., Gonzalez-Franco, M., Ofek, E. & Hinckley, K., The uncanny valley of haptics. (opens in new tab) Science Robotics. 2018.
Computer vision 
Graphics and multimedia 
Human-computer interaction