Getting Virtual with Dr. Mar Gonzalez Franco

Published

photo of Mar Gonzalez Franco - Researcher
Mar Gonzalez Franco – Researcher

Episode 4, December 13th, 2017 

Getting Virtual with Dr. Mar Gonzalez Franco

On today’s episode, neuroscientist and virtual reality researcher, Dr. Mar Gonzalez Franco, talks about her work in VR, explains how avatars can help increase our empathy and reduce our biases via role play, and addresses the misconceptions that exist between the immersive experiences of virtual reality and psychedelic drugs. 

Related:

Podcast Transcript

Mar Gonzalez Franco: Imagine you enter this new virtual body that you look down and your hands are these other hands. They move as you move perfectly, only you’re a different race, or a different gender. You can experience the life perspective of a different person from within.

Host: You’re listening to the Microsoft Research Podcast, a show that brings you closer to the cutting edge of technology research and the scientists behind it. I’m your host, Gretchen Huizinga. Today, we’re here with neuroscientist and Microsoft researcher Dr. Mar Gonzalez Franco. She’ll discuss her research in virtual reality, the importance of avatars, and she’ll clear up the misconceptions between VR and psychedelic drugs. That, and much more, on this episode of the Microsoft Research Podcast. Mar, you have a really interesting background. How did you end up doing research at Microsoft?

Mar Gonzalez Franco: I think in life, there are sort of cycles that you end up doing the same thing again. Like I started training as a computer scientist, mathematician, and then I became very interested by neuroscience. And then I came back to computer science. It kind of comes together. I would say there is always not a single road to get somewhere. And definitely you need to take it step by step. So I wouldn’t be here if I didn’t do my PhD with Mel Slater, and partially at MIT for sure. I mean, these – Microsoft Research really scouts for talent and in a wide spectrum, on a global scale. And you have to be there. And there is a part of being lucky to be in a very hot spot that is of interest to the company in a very specific moment.

Host: You told me that VR can trick the brain.

Mar Gonzalez Franco: Yes.

Host: And that people know it’s fake, but they have a strong experience of reality anyway. What’s good about that, and what’s bad about that, and what do we take away from that? It kind of scares me, but it’s kind of cool.

Mar Gonzalez Franco: Yeah, it’s very interesting, because we are providing a real-time stream of information to our senses. Mainly virtual reality has been based on vision, and humans are very visual animals. We believe what we see. And virtual reality provides that. It provides our congruent sensory motor stimulation. One of the experiments I did, you put on some sort of glasses – an HMD. With virtual reality, you put your hand on top of a table, and you see a virtual hand instead of your hand. And that hand gets attacked. And what I find is that the motor cortex gets activated in the same way that it would be activated for a real attack to your real hand. You really perceive this as your hand. I think this is very interesting, because it’s not only that you can enter a virtual reality. You can change your body in that reality.

Host: What are the implications of what you just said in terms of application? Outside the fact that you’ve observed that you can make people believe that a virtual hand is actually their hand?

Mar Gonzalez Franco: Some very interesting applications have to do with empathy. Imagine you enter this new virtual body that you look down and your hands are these other hands. They move as you move perfectly, only you’re a different race, or a different gender. You can experience the life perspective of a different person from within. Philosophers and psychologists have explored this role play. And this is role play to the maximum exponential. You’re basically within a different body.

Host: You’re talking about avatars now?

Mar Gonzalez Franco: Yeah.

Host: Whether it’s just a hand avatar or a full body avatar. And that’s the main component of virtual reality?

Mar Gonzalez Franco: I think avatars are becoming more important than ever with virtual reality. We experience reality and people experience ourselves through our bodies. It’s very easy to evaluate things from the perspective of our own body. And in virtual reality, we need to have a body, an avatar that represents us.

Host: If we can have realistic experiences through avatars, what other things might we be able to do through VR? How else might it help us?

Mar Gonzalez Franco: This is a very important topic in my research, the avatar that you embody. If you embody an avatar that is of a different race, you can reduce your racial bias. And this is some experiments that have already been done. There are other experiments that have shown that if you embody an avatar that is older, so it looks like you – it’s a lookalike avatar, just a little bit older – your financial decisions after you come out become more conservative. You’re more likely to save money. And not only this, even more traditional fears, like fear of heights, fear of flying – virtual reality provides a very controlled platform to perform therapy interventions. And during an intervention, people may develop a crisis, and you might want to take them out. In virtual reality, it’s simply a headset you remove it. You’re in an airplane; you’re out from an airplane. If you want to treat somebody who has a fear of flight, who has this problem, by putting them inside a real plane and flying them to New York, they might develop the crisis in the middle of the flight, and there is no way for you to stop it. In that scenario, it’s much more secure. So for anybody who is treating these kind of phobias, virtual reality is clearly a very good option.

Host: Is this happening already, or is this just something that you’re looking at one of the possible applications for VR in the future?

Mar Gonzalez Franco: No, people are already using it. Of course, there are only a few people that have these really, really extreme phobias that cannot be treated in a less immersive way. You could treat your phobias by going in the real world rather than in the virtual world. But it also allows you to treat your phobias from home. You have a device at home. You don’t need to go to a center to be exposed to this therapy. So it also democratizes a lot, these sort of treatments or interventions.

Host: Let’s talk about the illusory nature of VR for a bit. I read an article where the headline said, and I quote, “Microsoft says virtual reality could make you hallucinate in the same way as LSD.” That was a rather sensational version of what you actually said, I think. But can you talk a bit about how your work in VR is like and even unlike hallucinogenic drugs?

Mar Gonzalez Franco: When Jaron Lanier started the first virtual reality company, he was faced all the time like, oh, this going to be like an LSD trip? And of course the answer is no. But I’m going to go a bit more in detail, because I do think it’s legitimate for people to be concerned about how a technology is going to affect their life. Virtual reality, in the best form, provides a very strong illusory setup to create illusions. Illusions such as there is a glass on top of this table. But in reality, there is no such thing. With the reality, it has the same power. But this is an illusion in which you never lose your higher cognitive functioning. There is no chemical alteration of your brain. It’s purely sensorial. You’re seeing this glass of water. It looks super realistic. And you move around, and the glass is staying there. So at the end, it really feels like there is something there. And – but you take off the glasses, and there is nothing there. Compared to drugs, this person has never lost their higher cognitive function during this illusory state. And that changes completely the topic. Because higher cognitive function is the one that allows you to make conscious decisions. Decision-making processes, what’s right, what’s wrong. And because that’s never altered, you’re always aware that this is not real. Versus when people do drugs, that’s completely lost. So I say that these are very 2 different ways of experiencing reality.

Host: You’re really interested in the societal impacts of virtual reality. What are the best things about your work in VR?

Mar Gonzalez Franco: First of all, this democratization of the technology arriving people who have never been able to own any previous technological advancement. And now all of a sudden, they have something that includes them all. It’s like one ring to rule them all. And augmented reality and virtual reality are getting there. In fact, there is still a bit the fight on whether it’s going to take the form of a phone that you put on top, or it’s going to be a completely new device that you can render a phone in it. But it’s true that it’s going to be mobile in the sense that the battery, it’s going to be there. And you’re going to put it on, and render from first-person perspective. So that’s very exciting, because it’s going to have very big implications on society. It’s going to change many things of how we live.

Host: Give me an example of what you think that might be, one thing that could help us.

Mar Gonzalez Franco: Yeah, I think in particular this tendency we have to buy so many gadgets. That might actually go away, and we have one single gadget. This has not happened with any other device. Even we have tablets now, we still have a laptop, or we work on a desktop. If we’re talking about a device that is going to be able to embed them all, you’re really going to not have a desktop afterwards. You’re just going to have this device. That’s because you can render a computer in it, or you can render a tablet. Or you can render everything.

Host: Explain to me render. How does that work in virtual reality?

Mar Gonzalez Franco: In virtual reality, you have a display, a near-eye display, because it’s like a glass. So inside these glasses, you digitally print an image, just like you would do in a regular screen. It’s just that this screen, it’s always available for you. So inside this screen, you can print different things. Printing in digital is usually referred as rendering.

Host: What I hear you talking about is like a Google Glass kind of thing.

Mar Gonzalez Franco: It’s very different, because Google Glass was an annotation device. It would provide you information from the top, which is not blended with reality. This is not an annotation technique. This is a very different technological system. And this is with occurring with hollow lens. I can only imagine when we move forward, it’s something more like – instead of having a computer at home, or a TV, I have – my glasses know that this – this wall has a TV on it, and when I come home, the TV is there. I don’t need to put it back.

Host: But you’re looking through a device, yes?

Mar Gonzalez Franco: You’re looking through a device, like glasses, normal glasses.

Host: So does everyone in your family have to have them? Say like you have a family of 5. Everyone has to have the glasses in order to see the TV? The virtual TV?

Mar Gonzalez Franco: We’re working on this. Because it’s interesting to explore what each person will see. Because also using these glasses, I could do something very unique, which is seeing what you’re seeing. We’re seeing one in front of each other, and we have a chart in front of us, you’re seeing one perspective of the chart, and I’m seeing another one. What if we both could see exactly the same thing despite being in different positions? I mean, we start a very interesting topic of perspective.

Host: Wow, my mind is just reeling with what you’re talking about, and trying to imagine. Because my paradigms are so “right now,” right? My way of writing is using a keyboard and a screen. But you’re suggesting a whole new paradigm that we don’t even imagine.

Mar Gonzalez Franco: If we do it well enough, you will have to change nothing from your current interaction techniques. They will be embedded in the new technology.

Host: Let me just extrapolate here. I could have a device on and render a keyboard?

Mar Gonzalez Franco: Yeah. Or you will take your pen, physical pen. The pen will be recognized by these glasses, and you’ll write on a digital notebook.

Host: This is so sci-fi.

Mar Gonzalez Franco: Science fiction is a very good inspiration for science.

Host: What’s the biggest limitation you face right now? You’re sitting in Redmond doing research. And if you could say, hey, this would make my life way easier in what I’m trying to figure out here – what kind of things are impeding your progress?

Mar Gonzalez Franco: I think Microsoft is very good at giving all the resources you need to do whatever you want to do. Of course, you have to deliver, but that’s like in every other job. So I don’t think I have many – I would say it’s time. I don’t have time to do everything I want. But this is also good, because it gives you the unique opportunity to focus. You need to focus, and you need to pick fights that you want, the most important ones.

Host: And you actually said at one point that virtual reality can prove neuroscience.

Mar Gonzalez Franco: Yes. There are some theories that are only available because they have explored very particular pathologies, that happen very rarely in the normal population, maybe one of a million have something like that. Maybe it’s something related to epilepsy, removal of an epilepsy center that has detached your frontal lobe from the motor cortex, for example. So this is very unique scenarios, and there are people who undergo this pathology of the anarchic illusion, which they cannot control their hand at will. They know it’s their hand, but they cannot control it.

Host: And this is a real thing in real life?

Mar Gonzalez Franco: Yes.

Host: Anarchic hand?

Mar Gonzalez Franco: Yeah. It’s of one hand on it.

Host: Yes.

Mar Gonzalez Franco: No, but it’s like simple things like you’re eating a sandwich. I’m hungry. My hand knows I’m hungry. It picks your sandwich. So these people learn to – because the other hand usually is fully functional, they learn to hold their hand. So they just hold themselves.

Host: So they keep it from grabbing my sandwich?

Mar Gonzalez Franco: Yes, for example.

Host: Is that for real?

Mar Gonzalez Franco: Yes, it is. This is a real pathology. But this is a case-study type of thing. If you want to develop a real model of this, you might want to try to produce the same kind of illusion on healthy individuals. And then you can see what happens in their brain when this happens, and then compare it to what happens to a person who has this real pathology. Of course, these healthy participants, once they go out from VR, they are perfectly well. They don’t have any issue on their control of their body, just like when they enter virtual reality, they cannot control their body as they wish.

Host: But the point would be to see what comparisons you could make in a healthy brain versus an unhealthy brain, and how that might be able to treat this?

Mar Gonzalez Franco: Yes, and also to understand how the brain works in general, for both healthy and unhealthy. So there are 2 mechanisms in the brain that we use to control our experiences. The first one is a bottom up. It’s purely sensorial. Whatever we feel, we touch, we see, we integrate it in a motor sensory way. And that’s how we define reality. The other one is we have predictions. And sometimes they’re very strong. For example, when we feel the phone vibrates, and it didn’t. This is a top down illusion.

Host: Other people experience that too besides me?

Mar Gonzalez Franco: Yes.

Host: I’m not kidding.

Mar Gonzalez Franco: Yeah, it’s called…

Host: Phantom phone syndrome.

Mar Gonzalez Franco: Phantom phone, yeah. Yes.

Host: Get out.

Mar Gonzalez Franco: Yeah, so this experience is very sensorial. You would swear that you felt it. This is how powerful our brain is into manipulating our sensory input. And the interesting thing about the phantom phone, is that we become aware of it. But there are millions of predictions that are being rejected without even us being consciously aware of it.

Host: Wow.

Mar Gonzalez Franco: So in virtual reality, we can kind of start separating both, and seeing how they work one with the other. And it’s very interesting, because this – when you play with this, you learn a lot about the neuroscience, on things that are very hard to test in reality. So I think virtual reality is definitely a tool for neuroscience.

Host: Let’s say that you’re this generation of virtual reality and neuroscience exploration, if you will. What’s on the horizon for students coming out of university now?

Mar Gonzalez Franco: We have both in the US and in Europe very important big projects. There is the brain project in the US, and the human brain project in Europe, both of them with over a billion dollar funding, which is huge. They are mobilizing a whole community to finally get to understand the brain. And still, we don’t know it. And certainly even the people who are doing AI are very interested in understanding, because if you want to simulate something, you need to know how it works.

Host: What would you like people to know about the research you’re doing?

Mar Gonzalez Franco: I think in general, one of the things from a researcher perspective that is different from a product developer perspective, is the outreach. The fact that we can communicate, that you can do these types of things, or you could use technology to explore neuroscience, can improve dramatically a whole community of science. So how from one science you can transfer to another, so you are kind of becoming that bridge. I think when we work in research, we can transfer something, show that it’s useful for something else, and then empower a whole community into using it in a different way.

Host: How does doing research at Microsoft differ from doing research at say, a university, or another academic setting?

Mar Gonzalez Franco: The way Microsoft Research moves is very fast compared to academic research. For me, to pursue an idea, I need to convince my VP. So I need to pitch it. I have to think about it. But I don’t need to write a grant and then ask these very big organizations of a grant funding to evaluate it. And so it’s much faster. My ability to reach the people and to collaborate with them, it’s incredible. And I love it.

Host: Just as we go, just say one quick thing about what’s, what you’re the most passionate about right now in your work. I mean, aside from your outside life.

Mar Gonzalez Franco: I’m very interested about producing tactile sensations out of the body. I know this sounds very weird, but I’m creating this system in which you’re holding a virtual object, and you feel the virtual object being touched. But not directly on your hands. Somewhere else. It’s kind of difficult to explain it here, but this could mean that we can perceive touch in a similar way that we’re perceiving audio. And this is – I’m working with Christopher Berger on this. He’s Cal Tech. And we’re both super excited about it. And this has been my work during the summer. And this has a lot to do with haptics. So if you have very simple vibro-activator in your hands, and you’re able to perceive touch across the space, that’s even better what you can do in reality.

Host: Mar, you are fascinating. Absolutely fascinating. I want to have happy hour with you for real.

Mar Gonzalez Franco: Good. Anytime.

Host: To learn more about virtual reality and Dr. Mar Gonzalez-Franco, and the latest in Microsoft Research, explore our website at Microsoft.com/research.

[End of recording]

Continue reading

See all podcasts

Research Areas

Research Groups

Related labs