View Video (95meg)

I’m very interested in how tangible objects can be used in interesting ways to interact with information on screens. This video collects together a series of experiments on the use of a range of object prototypes. In making these, I imagined a screen in front of me (in some cases a standard size screen, in other cases a wall sized screen), and manipulated the various objects as if I was controlling and interacting with content on the screen. It was more an experiment in the affordance of the objects in relation to screens than thinking of specific applications.

This project was aided by the help of my graduate students Jonathan Jarvis, and Parker Kuncl.

The Acura Oracle project was initiated by MDP alumni Nikolai Cornell at George P. Johnson for the Acura stand at the international auto shows (including Detroit, Chicago, New York, and LA). I participated as a consultant in the initial creative brainstorming and technical definition. Bringing in Moto Development to create the multi-touch technology and optics, I coordinated Moto with the creative team, advised on interaction issues, and developed test projects to explore different interaction possibilities.

After the first year deployment at the auto shows, American Honda and GPJ lent the Oracle to the MDP for our summer term. I taught my Interactive Objects and Spaces class with the Oracle as one of our experimental design platforms. Several student and faculty projects came out of this, and IOS student Jonathan Jarvis then went on to intern at GPJ, implementing a version of his class project which became the Oracle application for the 2007/2008 auto show season. 

One of the Oracles has now been permanently donated to the MDP.

See also a page put up by Bill Buxton, one of the original people to implement multi-touch systems back in the ’80s.

The designinteract.com article is especially interesting in that it covers the entire process of the project, from concept through completion: www.designinteract.com/features/mirrors/ (site dead – instead please view a PDF of site rescued from the wayback machine at archive.org) 

For an overview of the project and video of people using it, see the dedicated site for it here: www.interactivemirror.net

.

An installation for Infiniti at the International Auto Shows, the Infiniti Interactive Mirror is a 3 screen interface that uses mirrored glass and rear projection to create a seamless large screen touch interface. George P. Johnson created the project with Nikolai Cornell (MDP alum, basing the project on his thesis project) as creative director. Nikolai worked with The Designory, MindFlood, and my company Commotion. 

On this project, I designed and built the sensor system that detects a person’s hand position in front of a flat surface without any sensors on the sides of the display or behind the user. In addition, I provided interactive consulting on the project.

Read an article on how the project was produced (rescued with the wayback machine – was originally published online at designinteract.com, but communication arts killed it).

[View with PicLens]

www.interactivemirror.net

Corporate Delicti was a group exhibition at Nucleus Gallery around the topic of disillusionment with corporate life. Collaborating with student Colin Owen, we built a system of interactive objects from an office – a copier, file cabinet, and fax machine. Painted gray, each object was made interactive by removing the guts and putting sensors and speakers inside.

The copier groaned when someone approached it, and when the “copy” button was pushed, a loop of copying sound played. With each subsequent push, a music track was added to the mix, building up a complete musical loop.

The file cabinet played more music tracks that synchronized with the copier sound. This track faded up as the user opened the drawer, bringing in more tracks the further the drawer was opened (typing, drums, etc.), In addition, occasionally when the drawer was opened, the sound of crows seemed to jump out of the cabinet and fly across the gallery space as the sound moved from a speaker inside the cabinet to speakers above it, and the across the room to speakers in the back of the gallery.

The fax machine played a loud and ominous sound when the handset was picked up. When the flash-hook on the phone was clicked, the fax machine played a computer voice reading a random segment of a contract stating the company owned everything created, throughout the universe. These contract excerpts used different computer voices and came out of different speakers each time they played.

[View with PicLens]

More pictures

10 minute documentation video

1st section at gallery opening, 2nd section staged demo, 3rd section walkthrough of entire show

This experiment allowed the band members to interact with the big video screens as they stood on stage. In the first project, Welcome to ZooTV, Bono selected a region, then the city, and then one of four versions of himself to welcome the audience to the show. The second project was BeatBox, and was run by Edge. He picked a song, and then played around with the beat by changing tempo and selecting between 3 different sound sets (including a zoo version with animal sounds for the kick drum, snare etc.). 

Apparently the band only used the projects on a couple gigs, as it turned out to be a bit too time-consuming for them to run it. If anyone has any stage pictures of them using it, please let me know. I think one of the shows where they used it was in Texas.

These projects were created using CDi technology developed by Philips and Sony. This was the first commercially released set top box that used CD media. My MusicWorks group produced several music titles for this medium. These were called CDi-Ready discs, and they played on normal CD players, but were the first music CDs ever to include multimedia content as well. Titles included James Brown, Mozart, Louis Armstrong, and Luciano Pavarotti.

Later, when I started Commotion, we went pretty far down the road on a proposal to create an interactive project with U2. It would have been developed on either CDi or the 3DO platform.

[View with PicLens]