Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation

(A) A blind user wearing the gear for our VR evaluation, including a VR headset and Canetroller, our haptic VR controller. (B) The mechanical elements of Canetroller. (C) Overlays of the virtual scene atop the real scene show how the virtual cane extends past the tip of the Canetroller device and can interact with the virtual trash bin. (D) The use of Canetroller to navigate a virtual street crossing: the inset shows the physical environment, while the rendered image shows the corresponding virtual scene. Note that users did not have any visual feedback when using our VR system. The renderings are shown here for clarity.

Traditional virtual reality (VR) mainly focuses on visual feedback, which is not accessible for people with visual impairments. We created Canetroller, a haptic cane controller that simulates white cane interactions, enabling people with visual impairments to navigate a virtual environment by transferring their cane skills into the virtual world. Canetroller provides three types of feedback: (1) physical resistance generated by a wearable programmable brake mechanism that physically impedes the controller when the virtual cane comes in contact with a virtual object; (2) vibrotactile feedback that simulates the vibrations when a cane hits an object or touches and drags across various surfaces; and (3) spatial 3D auditory feedback simulating the sound of real-world cane interactions. We designed indoor and outdoor VR scenes to evaluate the effectiveness of our controller. Our study showed that Canetroller was a promising tool that enabled visually impaired participants to navigate different virtual spaces. We discuss potential applications supported by Canetroller ranging from entertainment to mobility training.