Project Zanzibar

We began with a simple thought – what if we could blur the divide between the physical and digital worlds? What if you could play with physical toys, cards, and blocks, and watch your actions come alive on the screen?

Project Zanzibar is a flexible, portable mat that can sense and track physical objects, identify what they are, and allow you to interact through multi-touch and hover gestures.

Every object in the Project Zanzibar platform has a globally unique identifier (ID), which means data can be stored for each unique object. Collectible cards can gain points and XP, gaining value over time. Toys can learn their own history and tell unique stories. These natural ways of manipulating the world opens up a world of possibilities in terms of how people can play, learn and interact between the physical and digital world. Watch our working demo video below!

 

Play

Project Zanzibar allows kids to bring their toys into an interactive experience and watch them come alive through sound, visuals, special effects.

We observed kids spending hours making and sharing stop-motion movies, and envisioned using Project Zanzibar as a movie maker for kids to tell stories with their toys while seamlessly creating rich, animated videos.

In our movie maker prototype, kids can tell stories by manipulating toys and props on the mat to control corresponding graphical characters, previewing and capturing the result as a movie on the screen of a connected device. Camera and a light source objects allow the scene to be framed and lit. Additional props can be used to change the setting of a virtual scene. For example, a palm tree can change the virtual environment to a tropical setting.


The notion of screen-time is a growing concern amongst parents as children gravitate away from toys and towards mobile devices at an increasingly younger age, so Project Zanzibar offers the flexibility of playing with or without a screen. For example, the pirate toybox playset is a prototype that uses sound effects and audio dialogue to complement physical play.

The toybox includes figurines for different characters, such as pirates and naval officers. The combination and proximity of different characters on the mat triggers lines of dialogue from a story — for example, two pirates will banter if placed close to one another, while a pirate will hail an officer with a challenge. Through its unique ID, each pirate toy is unique, building on their narrative with each play session.

The sensing volume of the mat can be extended by some toy pieces, such as on the castle tower, allowing the mat to detect when a character is placed on top of it. The toys themselves can also be interactive – firing the cannon is accompanied by a booming sound, followed by splash when the cannonball lands in the sea, or a shout of if one of the characters is hit.

 

Education

We believe that Project Zanzibar is a powerful platform for education. With the Project Zanzibar mat, children are able to use physical objects and self-directed activities for a variety of subjects of learning. These activities are digitally enabled, providing the opportunity for intelligent feedback and guidance.

The Montessori-inspired letter plates exercise provides feedback as words are formed and tactile letter-shapes are traced. Paper overlays on the mat can provide additional context to the exercise.

We see a rich family of educational apps including spelling and simple coding exercises using blocks, with or without screens. By enabling kids to use their visual, physical senses, learning becomes more effective.

 

Gaming

Project Zanzibar provides a natural fit for extending many existing physical games into a digital space.

This prototype is a two player board game where players compete to destroy and defend their own spaceship. The game is played on a mat with an overlay depicting two spaceships segmented into various functional areas (helm, shields, weapons, and teleportation bays). The overlay also includes virtual buttons to fire weapons and complete a turn and areas where cards should be played.

Each player has a set of figurines of characters with different specializations (pilot, weapons officer, shield operator), and a deck of cards with additional offensive and defensive bonuses. Every turn players can play a card from their hand, move a figurine to a bay to boost ship function, and choose whether to charge or fire their weapons.

Once a turn has been completed, the effect of the player’s decisions and their outcomes are animated on-screen. Figurines have LEDs embedded in their base that change colour to indicate if a figurine has been placed in a valid or invalid location.

 

Features

We see Project Zanzibar as a platform that invites people to reimagine how they interact with digital systems. At its heart, this platform consists of:

  • A rollable mat that uses capacitive touch and NFC to track objects, their orientation and multi-touch gestures on its surface.

  • The Mat is able to uniquely identify objects if tagged with low-cost NFC stickers.
  • Custom NFC tag allow these interactions to be extended.
  • Extender tags allow objects to be identified when stacked above the mat.
  • I/O Tags provide digital inputs and outputs to an object such as LEDs and buttons without needing a battery.

  • The Project Zanzibar mat runs stand-alone with all sensing and tracking performed locally.
  • High-level interaction events are able to be sent to a connected device using USB or Bluetooth.

Project Zanzibar is Microsoft Research project, for more details on the project we invite you to read our paper titled Project Zanzibar: A Portable and Flexible Tangible Interaction Platform, in the paper section below.

We are very proud to have been awarded a Best Paper Award for CHI2018 (opens in new tab) – Come see us in Montreal April 23-26:

  • The Zanzibar demo on Monday night, April 23 @ 6-9pm
  • Our Zanzibar paper presentation on Tuesday, April 24 @ 2pm, Tangible Paper Session Room 513AB
  • Zanzibar demos at the Microsoft Booth throughout the conference