Tools and Techniques for Prototyping Future Interactions

January 31, 2005
Scott Klemmer | Stanford University

User interface tools aid in the design and development of interactive systems. The next generation of user interfaces is moving off the desktop: these emerging interfaces employ novel input techniques such as tangible, haptic, and camera-based interaction, access to vast information repositories and sensor networks, and information presentation to a wide range of devices.

Currently, developing this next generation of interfaces is difficult, time-consuming, and requires a high level of technical expertise. Consequently, only a small cadre of technology experts can currently build these UIs. The difficulty of technology development and lack of appropriate interaction abstractions make designing different variations of an application and performing comparative evaluations unrealistic. These difficulties echo the experiences of developing GUIs twenty years ago.

I’ll begin by covering my dissertation research on tangible interfaces: a wall-scale design tool for web site designers (The Designers’ Outpost), a handheld interface for oral historians (Books with Voices), and a toolkit for rapidly building tangible interfaces (Papier-Mâché).

Then, I’ll launch into three projects prototyping future interactions that are (or will soon be) underway.

Field Tools: Biology fieldwork generates a wealth of qualitative and quantitative information, an unstructured “bag of data,” requiring substantial labor to coordinate and distill. Through interviews with 15 biologists and analyses of 13 of their notebooks, we learned that notebooks serve as the primary record of observations, plans, measurements, and results. Our goal is to bring the field to the laboratory and the laboratory to the field. We introduce ideas to augment notebooks with two related goals: presenting notes alongside contextual information and using the notebook itself as a query interface.

d.tools: The confluence of computation (bits) and physicality (atoms) is one of the most important issues for both academic research and commercial interaction design. d.tools is a design suite for rapidly prototyping these systems. As an example, our goal is to be able to prototype an ipod in 20 minutes, and prototype 5 variations in an hour.

MultiMobile: In 5 years, mobile “phones” will have gigahertz processors and a gigabyte of RAM. Without research, however, the user experience of the mobile phone of 2011 will be nearly identical to the user experience of today: the interface is the bottleneck. In the interest of high-performance interaction with mobile devices, we propose the use of several communication modes in concert. Input modalities will include speech, stylus, physical keypad, tactile con-trols, computer vision, and location. Output modalities will include speech and non-speech audio, visual display, and haptic force feedback.

Speaker Details

Scott is an Assistant Professor of Computer Science at Stanford University, where he co-directs the Human-Computer Interaction Group.
Organizations around the world use his lab’s open-source design tools, and several books and popular press articles have covered his research. He is a co-recipient of the CHI and UIST Best Paper Awards, Microsoft Research New Faculty Fellowship, Sloan Fellowship, and NSF CAREER award. He received a dual BA in Art-Semiotics and Computer Science from Brown University, and an MS and PhD in Computer Science from UC Berkeley.