For most of us, a call to emergency services is a rare act, or one thankfully that we’ve never had to take. Anyone having had to make that call often will, in a calmer future moment, reflect on what transpired during it – the quality of the service, the professionalism and empathy of the call takers, and perhaps most importantly how information crucial to the outcome was conveyed. The communication aspect of emergency calls can mean the difference between life and death and 911 call takers are rigorously trained and experienced in guiding callers in order to extract and analyze information and send the needed assistance.
For most of its history, communication in emergency services calls has relied on the ability of the caller to convey information orally. In an age of ubiquitous video calling, this is going to change; over the coming years, emergency calling services in North America will begin to incorporate processes and technologies for video-based calling. The challenge isn’t that the underlying technology isn’t yet ready; it is. What isn’t yet clear is what unique benefits and challenges emergency video calling might bring.
Spotlight: blog post
GraphRAG auto-tuning provides rapid adaptation to new domains
GraphRAG uses LLM-generated knowledge graphs to substantially improve complex Q&A over retrieval-augmented generation (RAG). Discover automatic tuning of GraphRAG for new datasets, making it more accurate and relevant.
A team of thoughtful researchers at Microsoft Research, Simon Fraser University and the University of Calgary decided to examine and understand the issue of incorporating video in emergency calling services and gleaned significant insights on what might work – and not work – and why. In pursuing these insights, initially from the point of view of the professionals on the receiving end of these calls, they gained a deep respect and appreciation for the almost hidden world of the emergency call takers and what they do while defining the shape of a new research space and the issues involved. The resulting paper, The Benefits and Challenges of Video Calling in Emergency Situations (opens in new tab) was presented recently at the 2018 ACM CHI Conference on Human Factors in Computing Systems in Montreal. The research and findings will be of keen interest to everyone from emergency planning professionals, to mayors, city planners, federal agencies, technology companies and designers of mobile platforms and, perhaps especially, to lay people.
The idea for examining the ramifications of video calling in emergency situations emerged from conversations between Carman Neustaedter of Simon Fraser University’s School of Interactive Arts and Technology and Kenton O’Hara of Microsoft Research in Cambridge, United Kingdom. Neustaedter, along with his Simon Fraser colleague Brennan Jones, had long been interested in the use of mobile devices for video conferencing in the wild. With a shared interest collaborating on ideas for new uses of technology in health, Neustaedter and O’Hara lighted on the idea of the advent of video calling in emergency situations and what it might mean for people on both ends of the call – those calling for help and, relevant to this initial study, the call takers.
In setting out to understand the daily life and methods of the emergency call takers, the researchers spent a lot of quality time in the call takers’ world, sitting in on real-life emergency calls, conducting contextual interviews at three emergency response call centers in Canada and focusing on the practices of call takers and dispatchers. They came to realize that video calls could provide valuable contextual information about situations and could help overcome call taker challenges having to do issues such as information ambiguity, location, deceit and communication issues (such as poor English-speaking abilities). But they also learned that that video calling in emergencies has the potential to introduce issues involving control, information overload and privacy if systems are not designed well. The findings encourage deeper and expanded research and point to a design space rich with possibilities and potential problems that will need to be addressed.
“We saw a huge need but also a really challenging design space,” said Neustaedter.” “The call takers have a deep appreciation for the delicate balance that they walk – they are very good and have time-tested procedures for extracting the needed information from understandably panicked callers. At the same time, they see the technology that’s going to be coming to their work practices in the next few years and they don’t want that to mess up what works right now. They want their processes to get better but they’re not sure how it’s going to affect them and how it should work.”
“This paper acts as a call to the research community and industry to look at how to address the complexities and challenges of video emergency calling and how to make the most out of the potential opportunities that could emerge from introducing these systems.” – Brennan Jones
Controlling the call
The feedback gleaned by the researchers through their interviews with numerous call takers and dispatchers makes for some of the most fascinating reading in their paper. From the perspective of the call takers, 911 emergency calling is largely about benevolent control, of the caller and the situation. It’s about acquiring information in a specific order and manner that reflects the need for pertinent information while balancing situations in stressful and challenging circumstances.
The ability of call takers to take control of an emergency call is impressive. On oral calls, this is accomplished via spoken communication protocols that call takers are well trained and versed in employing. But the researchers found that video emergency calling will introduce a need for a level of control not seen before in existing literature around video chat or mobile video calling and it suggests that new protocols and sequencing will need to be created to prescribe what information should be shown and in what order over a video call, as well as how call takers should act on that information in order to ensure the best care while limiting liability issues.
Yet control in an emergency video call would be about much more than this; it requires deciding when video and how video should be used. The call takers who participated in the study had numerous concerns when it came to their own autonomy, that is, being able to choose what they would see on video. In a high stress job with frequent emotional pressures associated with empathizing with callers in very distressing circumstances, the desire for autonomy by the call takers is understandable. Despite the potential value for assessing scenes and gaining additional contextual information, video should then be considered secondary in nature for emergency calling, where it would augment audio calls, not replace them.
Essentially, emergency video calls must be considered very differently from conventional video calls; the notion of what is considered video might vary between live one-way video all the way to an audio call that involves images or video clips. Being able to fluidly move between these different visual modalities as part of a video call would be crucial, the researchers concluded.
Camera Work
Control of a 911 video call would also be about the specific camera work needed to capture and manage the information being conveyed. Getting value out of video during a 911 call might rely a lot on the ability of the caller to provide the necessary camera views allowing call takers to understand what they are seeing and to acquire needed information. There is no comparison between these scenarios and the everyday use of Skype to video call with family and friends; the specifics of what needs to be shown in the latter are far less critical. The researchers conclude that this suggests critical design implications to aid callers in performing camera work, perhaps even including a role for AI and automation.
The video option shows promise on a number of levels. Yet the researchers are aware that the presence of video can change the nature of conversations, make them less efficient. Again, the findings suggest a need for a range of visual options in 911 video calls.
Had any of the researchers ever had to call 911? “I have,” said Neustaedter. “I could really empathize with what the call takers in our study were hearing because I’d been in those situations before. It’s one of those things in life that you don’t forget; you have rich memories of the experience and you can imagine how it could change with technology. This memory ability was invaluable in our research involving the callers.”
Next Steps
The researchers recently completed a study that looks at video emergency calling from the point of view of emergency callers and will present their paper, Caller Need and Reactions to 9-1-1 Video Calling for Emergencies (opens in new tab) at the Designing Interactive Systems Conference in Hong Kong in early June.
Neustaedter spoke of the important roles to be played by multiple entities, from national and local governments, to schools and technology companies on the journey to adoption of beneficent emergency video calling systems. Indeed, because smart phones are corporate products, platform and device companies like Microsoft, Apple and Google will almost certainly be crucial to the effort.
That this research is getting a lot of attention attests to the serious role emergency services play in communities throughout the world and the faith that all of us, regardless of our background or means must place in such services when we least expect it. This importance is reflected in the passion the researchers exude for this project and Microsoft Research is looking forward to an emerging research space accompanied by very interesting stories and thoughtful technologies. As Neustaedter put it eloquently, “We don’t call 911 very often thankfully. But when we do, it’s super important to get it right.”
Human-computer interaction