Mixed Reality Technology in Healthcare | Microsoft Enterprise
 |
Focus on: Microsoft in Health Digital Transformation |
I attended an internal event recently where Alex Kipman, technical fellow and chief evangelist for HoloLens, outlined his vision for the potential of mixed reality technology. It gave me chills. He opened with “We are standing together on the threshold of the next revolution in computing” and went on to add that “Computing will allow us, for the first time, to renegotiate our very contract with reality”. I found this moving because, for the past two years since the release of Microsoft HoloLens, I have seen incredible innovation flooding in from the health community. This response is distinctly different from past reactions in a few key ways.
- It’s a global phenomenon right out of the gate. I have seen examples from Brazil to the US, Norway to Australia, Japan to Canada. Hardly a week goes by without a new example cropping up somewhere. Interestingly, some of the less regulated health markets are moving faster and really pushing the boundaries of what is possible.
- It’s led by clinicians. New technology in healthcare tends to be embraced first by technologists, who then translate for clinicians how this could make a difference. By contrast HoloLens seems to be picked up directly by clinicians, and they are right in the driving seat of many of these examples. While Microsoft is working directly with some of these individuals and organizations, the movement has its own momentum and is proceeding independently.
- Mixed reality is going mainstream. Historically new technologies in health are treated with skepticism or cautious enthusiasm. This is the curve I envisaged for HoloLens too, but I have seen it make the leap from the lab to the clinic much sooner than expected.
With this as a backdrop, I think it’s timely to start sharing this story more broadly. I’m looking forward to the upcoming Asia Pacific MedTech Forum in Singapore where I’ll have the opportunity to share this story with a select audience. Knowing not everyone will be able to attend however, I’m keen to document some of what I’ll talk about so you can see the timeline of how this fascinating mixed reality technology is progressing.
From the very moment we introduced HoloLens to the world in 2015, health was front and center. The scenario that Case Western Reserve University in the US chose to focus on was anatomy training for medical students, and the feature video still remains one of my favorite examples of how mixed reality is a game changer in health. Subsequently other institutions have launched similar initiatives, including St George’s University in Grenada collaborating with SphereGen to create Living Heart, freely available in the HoloLens store. The ability to visualize complex three-dimensional structures in space, to see where the pancreas sits in relation to the liver and stomach, to understand how the cardiac valves move as the heart pumps, it totally changes understanding.
CAE in Canada took this a step further with their VimedixAR application. Instead of abstractly floating holograms in space, they overlay holograms on top of more traditional simulation mannequins. This is a long way from how I learned medical procedures and surgical techniques, the “see one, do one, teach one” philosophy. With this solution, you can visualize where the where the cardiac catheter is heading in relation to the beating heart, or how the structures of the thorax correspond during a trans-esophageal echocardiogram.
The first cross-over to more mainstream patient management I became aware of was the work Sopra Steria were doing with the University Hospital of Oslo in Norway. They were using holograms to plan surgery. Specifically, liver tumour resections and pediatric cardiac surgery. Ironically, the pediatric cardiac surgery modelling replaces 3D printing, which in itself was a recent innovation at the University. Sopra Steria received the Microsoft 2017 Health Innovation Award, and you can see their work here. Subsequently, I’ve seen a raft of examples of HoloLens being used intraoperatively, from reconstructive spinal surgery in Brazil or in the US with Scopis, to HoloEyes in Japan, and even neurosurgery at Duke University featured by CNBC.
I’m aware of many parallel initiatives to make medical image DICOM files and their 3D reconstructions directly visible on the HoloLens, but I thought the University of Maryland’s approach to virtual ultrasound opens up so many practical applications for democratizing and mobilizing this mixed reality technology.
The final example that I want to share has caused waves of attention around the world. It hails from Australia, where Silverchain are working on a holographic doctor service. I got to speak to CEO Chris McGowan over dinner recently at the Envision conference and can report that innovation is alive and well with this kind of leadership.
I return to where I started, with Alex Kipman’s bold claim that we are on the threshold of a computing revolution. From my perspective, the revolution is well underway. I do hope to share this journey with all of you as it progresses, and look forward to more examples that inspire and amaze me!
