This free collection of software makes it easy integrate all kinds of media with microcontrollers like the Arduino. Using a simple drag-and-drop interface, you can create interactive projects that combine sensors, video, text, graphics, sound, lighting, motors and more. These projects can be created quickly, without programming, using the smart widgets included with the Toolkit.

I initiated the project in 2004, and have continued developing it since. It is used all over the world by schools, individuals, and agencies.

By the time I got some students to sign up, and an approach approved, we had just one week to make the entire project.

The project consists of six plywood panels mounted to the wall, with separate slideshows running on each panel. Because the panels are different distances from the wall, there’s a dimensional effect created. The idea was to break the normal flat rectangle of projection and create an installation that felt more like a physical part of the space. In addition to the randomly playing slideshows on each panel, we created a simple interactive feature so if someone walked up to the wall, a flourish of motion graphics would appear unifying all the panels, then fading into a photo of the front of the museum spread across several panels.

The project was a collaborative design and build between myself and three students: Brooklyn Brown, Manny Darden, and Rubina Ramchandani. The design approach was partially inspired by some of Manny’s thesis work. For software, we used my NETLab Toolkit with a new SlideShow widget I developed that runs each of the slideshows - the entire project has no ActionScript, using only the toolkit widgets. Images were placed in folders, and each SlideShow widget played a set of images from these folders in a random order. Two projectors were used to get a wider display (2500 pixels), and these were fed by a video splitter out of a MacBook Pro (this way the Flash movie played across both projectors). An Infrared proximity sensor was used to detect someone in front of the wall, and this started the playback of five different video streams across the different panels.

Video of the interaction

Video of the overall project

A capture of the slideshow

The challenge with this project was to make access to thousands of database entries easy to use for school administrators, researchers, parents, and the press. By creating a clean and clear visual and interactive system, users can explore the data in a very direct and productive way.

Working with Liz Burrill and Jamie Cavanaugh, I developed the interaction design and functional capabilities, and then built out the database and software. The system allows users to do incremental searches based on county and/or school name, showing immediate results as the user types. When a school is selected, it animates into the Schools of Interest List, building the user’s personal list of schools to analyze. The list can then be customized by narrowing the set of indicators to focus on, and then printed or downloaded as an Excel file. Alternatively, the user can select any single school, and analyze it over several years to see how performance has changed.

This project was aided by the help of my graduate students Jonathan Jarvis, and Parker Kuncl.

View Video (95meg)

In collaboration with Liz Burrill (visual design), and Jamie Cavanaugh (technical production & animation), I developed the interaction design and wrote the software.

I participated as a consultant in the initial creative brainstorming and technical definition. I brought in Moto Development to create the multi-touch technology and optics, coordinated Moto with the creative team, advised on interaction design issues, and developed test projects and tools exploring different interaction possibilities. In the summer of 2007 GPJ loaned an Oracle for use in my class, and student Jonathan Jarvis made demos that became the design basis for the second version that was at the 2008 auto shows.

Video after the break

One of the Oracles has been permanently donated to the MDP

See also a page put up by Bill Buxton, one of the original people to implement multi-touch systems back in the ’80s.

Some of these ideas ended up in more complete projects, including the Acura Oracles and A+D Museum projects. For the Oracles, the idea of generating bits of information from a database (as in the Words project) plays out as the user moves their finger across the touch table. In the A+D project, the ideas of simultaneity, juxtaposition, and randomness play out as the slideshows display. We also creating a dynamically updating system, inserting live images from the event.

Detailed explanations and videos after the break.

• Words – This project is on the newecologyofthings.net website, and was part of the New Ecology of Things transmedia publication. It displays a cascading waterfall of phrases from the NET book’s glossary. The user can stop any word by hovering over it, and show a google search by clicking on a word.
  • Concepts: serendipity, randomness, simultaneity, center/periphery, passive interaction, associative linking
  • Link: newecologyofthings.net/words/
• Topic Explorer – This project takes a different approach to the presentation longer form written material. There are two parts to it. One uses political statements on the economy and the environment from two presidential candidates and allows the user to compare them by reading directly, or exploring the texts through a text waterfall or specific keywords. The second part takes an article from Metropolis Magazine and reworks it with a similar approach, except adding visual excerpts (small rectangles randomly clipped out of the photos in the article) to the waterfall. This project intentionally uses very large displays (especially for its time) to take advantage of the amount of information that can be presented simultaneously.
  • Concepts: excerpting, serendipity, juxtaposition, randomness, center/periphery, passive interaction, wide-bandwidth interaction
  • Link: See below for videos
• Text Facets – This project explores showing different facets of a body of information. Again drawing on the policy statements of a presidential candidate, it creates a way for the user to select topics the want to view, and makes it easy to juxtapose different policies, while compressing or expanding them.
  • Concepts: faceting, remixing, filtering
  • Link: See below for videos
• 3D Interfaces – Since almost all information is presented in a very flat manner, I wanted to see what would happen if an interface took advantage of our innate abilities to understand the world spatially. Would this make seeing different ideas make more sense – i.e. a literal kind of faceting across different sides of a cube? Or a more abstract 3D shape that contains different parts of a body of information?
  • Concepts: faceting, 3D manipulation
  • Link: See below for videos

On this project, I provided consulting on interaction design and approaches for the project, and designed and built the sensor system that detects a person’s hand position in front of a flat surface without any sensors on the sides of the display or behind the user.

Read an article on how the project was produced (rescued with the wayback machine – was originally published online at designinteract.com, but communication arts killed it).

[nivo source="nextgen" gallery=2 effect="slideInRight"]

www.interactivemirror.net

Video of project after the break

The copier groaned when someone approached it, and when the “copy” button was pushed, a loop of copying sound played. With each subsequent push, a music track was added to the mix, building up a complete musical loop.

The file cabinet played more music tracks that synchronized with the copier sound. This track faded up as the user opened the drawer, bringing in more tracks the further the drawer was opened (typing, drums, etc.), In addition, occasionally when the drawer was opened, the sound of crows seemed to jump out of the cabinet and fly across the gallery space as the sound moved from a speaker inside the cabinet to speakers above it, and the across the room to speakers in the back of the gallery.

The fax machine played a loud and ominous sound when the handset was picked up. When the flash-hook on the phone was clicked, the fax machine played a computer voice reading a random segment of a contract stating the company owned everything created, throughout the universe. These contract excerpts used different computer voices and came out of different speakers each time they played.

10 minute documentation video - 1st section at gallery opening, 2nd section staged demo, 3rd section walkthrough of entire show

More pictures

As part of my work leading the group under Joy, I worked with a range of musicians and designers, interfaced with product design companies IDEO, Astro Studios, and engineering company Moto. I also worked with Neal Bhadkamkar and others on business planning, creating detailed Bill of Materials for proposed products, and researching potential commercialization options.

It was at Interval that I became convinced that tangible interaction was the key to future forms of innovation.