MobileFusion: Research project turns regular mobile phone into 3D scanner

Published August 24, 2015

Share this page

By Allison Linn, Senior Writer, Microsoft Research

A new Microsoft Research project lets people to create high-quality 3D images in real time, using a regular mobile phone, with about the same effort it takes to snap a picture or capture a video.

“What this system effectively allows us to do is to take something similar to a picture, but it’s a full 3D object,” said Peter Ondruska (opens in new tab), a Ph.D. candidate at Oxford University who worked on the project while he was an intern at Microsoft Research.

The researchers say the system, called MobileFusion (opens in new tab), is better than other methods for 3D scanning with a mobile device because it doesn’t need any extra hardware, or even an Internet connection, to work. That means scientists in remote locations or hikers deep in the woods can capture their surroundings using a regular cell phone without a Wi-Fi connection.

“Everything happens on the phone itself,” said Pushmeet Kohli (opens in new tab), a principal research scientist with Microsoft Research who also worked on the project.

The scans are high-quality enough to be used for things like 3D printing and augmented reality video games.

Shahram Izadi (opens in new tab), a principal researcher who also worked on the project, said he imagines people using a tool like this to take a 3D scan of something they see on vacation, such as the Eiffel Tower, and immediately sharing it with friends or family.

He said a person might also use it to take a 3D scan of something they wanted to sell online, such as a vase or a lamp, and post that, instead of the more conventional picture or video.

“This is really about the accessibility and ubiquity of 3D scanning,” Izadi said.

The researchers will present MobileFusion in early October at the International Symposium on Mixed and Augmented Reality (opens in new tab).

Kohli and Izadi had previously worked on a similar project, called Kinect Fusion (opens in new tab), which lets people build 3D models of their houses, offices or even themselves. However, that system requires a PC and other gadgets, making it difficult to do any 3D scanning on the go.

When Ondruska came to the lab, Kohli said they were curious to see if they could “go to the next level, in the sense of usability.”

They found most mobile devices had become powerful enough that they could build a 3D-scanning system just using the computational power found on a regular mobile phone.

“The great starting point was to take a sensor that everyone has in their pocket, which is the camera you have on your mobile phone,” Izadi said.

The researchers then developed an algorithm that allowed the camera to act as a 3D scanner, using a technique of taking multiple images that is similar to how the human eye works, Izadi explained.

Currently, the researchers are working on making sure the system works with all types of mobile devices, including Windows Phone, Android and iPhone devices. Izadi said they hope to eventually make it available to the general public in some form, but there are no firm plans right now.

Related: