By George Thomas, Jr., Writer, Microsoft
Advances in machine learning and artificial intelligence aren’t just improving the quality of information available to us — they’re also making it easier to interact with the devices we use to retrieve and manipulate the information.
At ACM CHI 2016 (opens in new tab), the premiere conference on human-computer interaction (opens in new tab), Microsoft will present research advances that could vastly improve user interaction in a number of ways and contribute to the company’s ambition to deliver more personal computing experiences. The conference, which takes place May 7 to 12 in San Jose, California, offers a glimpse into the near-future, when we will be able to do things like use mobile devices in new ways and more easily navigate through virtual environments.
Spotlight: Event
A touch of virtual reality
Advances in virtual reality have thus far mainly been in the realm of optics, rendering and audio technologies. But improving haptics — the sense of touch — remained elusive until now.
The key objective in virtual reality is establishing a sense of presence. It’s easy to suspend disbelief if the environment looks real, but when you reach out to touch a virtual object and your hand goes through it the illusion is shattered. And the interaction itself changes the virtual scene, requiring the whole environment to be redrawn or rebuilt.
A new framework called haptic retargeting (opens in new tab) essentially “hacks” human perception and leverages the dominance of vision when our senses conflict. This allows for the development of much more complex virtual environments that can have many more virtual objects with which to interact.
Andy Wilson (opens in new tab), a principal researcher at Microsoft, said the long-term implications of this research will be limited only to the imagination of designers and developers.
The research is detailed in Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences (opens in new tab) and was developed in collaboration with the University of Southern California and the University of Waterloo.
Precognitive touch screens
Imagine a mobile device that intelligently anticipates your intended action even before you touch the screen.
That’s what’s being presented in Pre-Touch Sensing for Mobile Interaction. Ken Hinckley (opens in new tab), a principal researcher at Microsoft who led the project, said the research is based on a whole different philosophy of interaction design.
The research uses the phone’s ability to sense how you are gripping the device as well as when and where the fingers are approaching it.
“It uses the hands as a window to the mind,” Hinckley said.
By allowing the interfaces to adapt to you, on the fly, they are always tailored to the specific context of how you are currently holding or using your phone.
“I think it has huge potential for the future of mobile interaction,” he said. “And I say this as one of the very first people to explore the possibilities of sensors on mobile phones, including the now ubiquitous capability to sense and auto-rotate the screen orientation.”
Learning to code without known variables
As probabilistic programming (opens in new tab) gains popularity in data analytics, so does the need to make writing the programs more accessible to students and developers.
Now there’s a tool that combines a conventional text editor with chart visualizations that, in initial testing, reduced the time, keystrokes and deletions required for the typical tasks novices do when learning to program.
In standard computer programs, variables have set values. But in probabilistic programming, a method of reasoning backwards that lends itself well to machine learning tasks, variables can have uncertain values. For example, a variable could have a value “some-number-between-1-and-100.” This lets you learn about the value of variables without having to write a machine learning algorithm.
The research, A Live, Multiple-Representation Probabilistic Programming Environment for Novices, was conducted in collaboration with the University of Cambridge.
Best Paper awards: “You’ve got mail. Somewhere.”
Microsoft researchers and engineers received seven Best Paper awards at the CHI conference, including Finding Email in a Multi-Account, Multi-Device World (opens in new tab).
The research, done in collaboration with University College London, found that while email remains an integral part of our everyday lives, the way we access it in the future may change drastically, evolving beyond the use of today’s email applications.
(opens in new tab)Abigail Sellen, principal researcher at Microsoft’s Cambridge, U.K. lab, contributed to research about email that received a Best Paper Award.
These days, we aren’t just accessing email across multiple devices. The researchers found that we approach our personal and professional email from different perspectives, which in turn affects how we interact with our email.
While today’s email applications can be designed to better suit these differing habits, the future of email may be in how it is integrated into more flexible applications and services.
Fulfilling Microsoft’s mission
These are just a few samples of the variety of research Microsoft is presenting at CHI 2016. Microsoft is a sponsor of the annual conference in part because of its long history of work in human-computer interaction, notes Mary Czerwinski (opens in new tab).
A research manager in Microsoft Research’s Visual and Interaction Research Group (opens in new tab), Czerwinski said the conference “has been one of the premiere venues for disseminating research in the area of human-computer interaction, and has been one of the best vehicles for Microsoft’s mission: To empower every person and every organization to achieve more.”
Other Best Paper awards:
- Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media
Microsoft contributors: Munmun De Choudhury, Emre Kiciman, Mark Dredze - Augmenting the Field-of-View of Head-Mounted Displays with Sparse Peripheral Displays (opens in new tab)
Microsoft contributor: Hrvoje Benko - Chain Reactions: The Impact of Order on Microtask Chains (opens in new tab)
Microsoft contributors: Shamsi T. Iqbal, Jaime Teevan - FlexCase: Enhancing Mobile Interaction with a Flexible Sensing and Display Cover (opens in new tab)
Microsoft contributors: David Kim, Sean Fanello, Shahram Izadi - Going Dark: Social Factors in Collective Action Against Platform Operators in the Reddit Blackout (opens in new tab)
Microsoft contributor: J. Nathan Matias - FingerIO: Using Active Sonar for Fine-Grained Finger Tracking (opens in new tab)
Microsoft contributor: Desney Tan
Additional Microsoft Research papers from CHI 2016:
- The Tyranny of the Everyday in Mobile Video Messaging (opens in new tab)
Microsoft contributors: Sean Rintel, Richard Harper, Kenton O’Hara - A Comparative Evaluation on Online Learning Approaches using Parallel Coordinate Visualization
Microsoft contributor: Bongshin Lee - Supporting Collaborative Writing with Microtasks (opens in new tab)
Microsoft contributors: Jaime Teevan, Shamsi T. Iqbal, Curtis von Veh - “With most of it being pictures now, I rarely use it”: Understanding Twitter’s Evolving Accessibility to Blind Users
(opens in new tab)Microsoft contributors: Meredith Ringel Morris, Catherine Yao - On-Demand Biometrics: Fast Cross-Device Authentication (opens in new tab)
Microsoft contributor: Christian Holz - Things We Own Together: Sharing Possessions at Home (opens in new tab)
Microsoft contributor: Siân Lindley - Integrating the Smart Home into the Digital Calendar (opens in new tab)
Microsoft contributor: David Kim - Challenges for Designing New Technology for Health and Wellbeing in a Complex Mental Healthcare Context (opens in new tab)
Microsoft contributors: Anja Thieme, Siân Lindley - “Why would anybody do this?”: Older Adults’ Understanding of and Experiences with Crowd Work (opens in new tab)
Microsoft contributor: Meredith Ringel Morris - Setwise Comparison: Consistent, Scalable, Continuum Labels for Computer Vision (opens in new tab)
Microsoft contributors: Advait Sarkar, Cecily Morrison, Yordan Zaykov, Abigail Sellen, Sian E. Lindley - Neurotics Can’t Focus: An in situ Study of Online Multitasking in the Workplace (opens in new tab)
Microsoft contributors: Shamsi T. Iqbal, Mary Czerwinski, Paul Johns - Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media (opens in new tab)
Microsoft contributor: Emre Kiciman - Augmenting the Field-of-View of Head-Mounted Displays with Sparse Peripheral Displays
Microsoft contributors: Robert Xiao, Hrvoje Benko - Expressy: Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions (opens in new tab)
Microsoft contributor: Steve Hodges - Impact Chain Reactions: The of Order on Microtask Chains (opens in new tab)
Microsoft contributors: Jamie Teevan, Shamsi Iqbal - SnapToReality: Aligning Augmented Reality to the Real World (opens in new tab)
Microsoft contributors: Benjamin Nuernberger, Eyal Ofek, Hrvoje Benko, Andrew D. Wilson - Email Duration, Batching and Self-interruption: Patterns of Email Use on Productivity and Stress (opens in new tab)
Microsoft contributors: Shamsi T. Iqbal, Mary Czerwinski, Paul Johns - Toward a Learning Science for Complex Crowdsourcing Tasks (opens in new tab)
Microsoft contributors: Ece Kamar, Eric Horvitz - Making Sense of Temporal Queries with Interactive Visualization
(opens in new tab)Microsoft contributors: Danyel Fisher, Robert DeLine, Mike Barnett, Badrish Chandramouli, Jonathan Goldstein
Related: