Posted by
One day, a woman named Lisa is walking her dog, Lucky, when she meets an elderly neighbor. Suddenly, the neighbor falls unconscious and stops breathing.
Microsoft Research Blog
Microsoft Research Forum Episode 3: Globally inclusive and equitable AI, new use cases for AI, and more
In the latest episode of Microsoft Research Forum, researchers explored the importance of globally inclusive and equitable AI, shared updates on AutoGen and MatterGen, presented novel use cases for AI, including industrial applications and the potential of multimodal models to improve assistive technologies.
What would you do in this scenario? These days, many people’s initial reaction would be to reach for their mobile phones and seek help. But today’s search engines don’t offer pertinent assistance for such a situation. You can go to Wikipedia or the Mayo Clinic site, but those resources are designed for relaxed information seeking, not for urgent settings such as the one above.
How could search help? Working with a collection of collaborators, Nina Mishra (opens in new tab) of Microsoft Research aims to find out.
Mishra is the co-author—along with Microsoft Research colleagues Eric Horvitz (opens in new tab), Samuel Ieong, and Ryen White (opens in new tab)—of a paper entitled Time-Critical Search that will be presented at the Association for Computing Machinery’s 37th annual international conference of the Special Interest Group on Information Retrieval (opens in new tab) (SIGIR), to be held on Australia’s Gold Coast from July 6 to 11. And, yes, Mishra can offer a way to help Lisa to assist her stricken neighbor.
First, Lisa can ask her phone how to perform cardiopulmonary resuscitation (CPR). Mishra and her co-authors developed methods that provide search engines with the ability to recognize time-critical queries. These methods enable Lisa’s phone to detect the urgency of the query and suggest that she call 911 immediately and then follow step-by-step instructions. While the ambulance is on its way, the phone advises Lisa on valuable assistance to provide. Using the location sensor on the phone, Lisa finds an automated external defibrillator nearby, and the phone suggests that she ask someone to retrieve it. The phone then tells Lisa to place the phone directly on the neighbor’s chest and to perform chest compressions directly on the phone. The accelerometer in the device explains to Lisa the correct depth of compression and how quickly they must be performed. Lisa follows the instructions, an ambulance pulls up, a life is saved.
Collaborative Project
This research project, which spans Microsoft Research’s Silicon Valley (opens in new tab) and Redmond (opens in new tab) labs and includes collaboration with the Division of Emergency Medicine at the Stanford School of Medicine, examines how people use search engines when in time-critical situations—and the opportunities for extending search to consider the urgency of queries.
The usage patterns are gleaned from analysis of mobile search logs, including sensor data available from mobile devices. Those patterns characterize the use of search engines for time-critical situations and enable the researchers to develop models to predict urgent informational needs, given a query and a diverse set of features with topical, temporal, behavioral, and geospatial attributes.
The methods have been found to have the ability to detect medical emergencies with a high degree of accuracy and, having done so, can deliver targeted first-aid information. Thus, your mobile phone could prove to be an essential asset when you are confronted with a medical emergency.
“The paper in SIGIR discusses how a search engine can detect urgency,” Mishra explains. “For the other parts of the scenario, we have initial prototypes, but the work needs further development and evaluation.”
To develop techniques for providing relevant, understandable information, she is working with Microsoft Research colleagues Mark Manasse (opens in new tab), Jean-Philippe Martin (opens in new tab), Andreas Nowatzyk, and Li Zhang (opens in new tab), as well as with Greg Gilbert and Alexei Wagner of the Stanford Emergency Medicine group, which is providing invaluable domain knowledge.
The settings in which this research could have lifesaving potential are not limited to the CPR scenario above. Others could include queries such as:
- Am I having a stroke?
- How can I stop severe bleeding?
- A baby is choking!
(opens in new tab)The CPR scenario, though, is interesting because bystanders often perform chest compressions incorrectly. The technique the researchers are pursuing uses the accelerometer on a mobile phone to deliver real-time feedback to the person performing the chest compressions. The phone can offer auditory and visual feedback to direct the user to identify the correct depth needed for an effective chest compression (two inches) and how quickly the compressions should be performed (100 times per minute). The phone also vibrates at the rate of 100 times per minute, which can be invaluable amid chaotic situations in which verbal cues can be difficult to hear.
While work remains to be done, Mishra’s sights are focused on using the research results from her and her colleagues to deliver real-world help at critical moments.
“Going forward,” Mishra says, “we envision using the richer signals of a smartphone to assist the next person in need of help.”