Introduction

Interactive stories are often thought to be a series of choices made by the active viewer or participant. Go left, pick up a key, fall in love, steal a car, etc. Besides some challenges in terms of plot, character development, and consistent theme, this approach has fundamental problems related to a mathematical problem.

The Combinatoric Explosion

If the story is seen as a series of binary choices–do this or do that–the range of possible outcomes becomes very large in just a few choices (and the problem is made much worse if the number of choices is greater than two at each decision point). For example, after only six choices, the number of different story lines is sixty four, as can be seen in the drawing to the left. If each choice leads to a unique story line (in other words, the different choices do not result in common outcomes), the space requirements and production costs for such an interactive expression become prohibitive before the participant spends ten minutes with the product. Imagine trying to write, videotape, and assemble 64 separate story lines.

Abandoning The Traditional Narrative

The combinatoric explosion described above is a consequence of thinking in terms of the traditional narrative. But if the interactive expression is seen in different terms, the problem becomes more manageable. Imagine that instead of following the straight line of the narrative, the participant travels through a story space. The story space is composed of all the different locations, events, times, and even moods carefully created by the author. The process of moving from one place in the story space to another is determined by their “distance” from one another, and the intervening story places.

The traditional narrative can be achieved in this story space by following a straight line. But a different story line can be achieved by traversing a different path through the story space. In addition, different beginnings and endings can be achieved by entering and exiting the story space in different places.

In this form, the concept of the story space can be useful in constraining the combinatoric explosion. The author creates only those story places within the space that help support the idea being expressed, and size of the production can be contained and made producable. But one can think of the above two dimensional story space as essentially static. To move to a more dynamic model, a third dimension can be added. If the participant makes a significant change to the environment, rather than just traveling through it, they might jump to a different “plane”. For example, by choosing to help the homeless person find a place to sleep, rather than ignoring them, the participant may jump to a different or altered plane in the story space as a consequence of their choice.

While this dimensional approach could lead to the same combinatoric explosion (by creating too many planes), the author controls the explosion by creating only the places and paths-between-places she wants the participant to explore. This is the power of Story Space model. Rather than thinking in terms of a straight line with a beginning and end, the author thinks of a story space through which the participant moves.

Ways of “Moving” in the Story Space

The participant can be allowed to move through the story space in several different ways:

• Changing locations or times. For example moving from the living room to the bedroom, or from Tuesday to Wednesday.
• Changing perspectives. The participant can look through the eyes of different characters, or the same character but with different attitudes. One could also be a passive observer rather than one of the characters.
• Changing scope. Elements of the story can be seen in great detail, or in broad strokes.
• Changing modes. One could read text, see pictures, or hear sounds.
• Reordering the sequence. Parts of the story can be viewed in different orders.
• Altering the space via practical or moral choices. The participant can send the love letter, or deem the protagonist to be an artist.
• Passive interactivity. The story can change itself, either randomly, or based on the intentions of the author.

I think passive interactivity, while facetiously named, is critical to the success of Interactive Expression. First, participants will soon grow weary of mundane choices and may prefer to sit back and let the story evolve on its own. One night, you might watch the new western and see the goings on down in the valley. The next night, you watch the same western, but see the murder in the mine shaft. Second, if the participant is to be lead to a particular idea by the author, the story may well need to draw the participant along, closing off certain choices, opening up others, and imposing yet others. Even if several different ideas are being presented, and they depend on the path taken by the participant, the story may need to draw the participant to one of those ideas. At a minimum, a certain amount of motivation, fun, change, randomness, and action is likely to be needed, and this can be provided by passive interactivity.

Afterthoughts: Dimensional Story Space

1998
In 1992, I co-founded a salon and newsletter called Interactive Expression, where we explored many of the central issues of multimedia as an art form. For the newsletter, I wrote a piece called that explored some of the problems with interactive story telling. Someone recently sent me an email about it, and it prompted me to look back and think some more about multimedia with the perspective that six years has given me. It is difficult to express something meaningful if the authorial voice is destroyed by the form of the text. And to be honest, I haven’t seen a fully successful interactive text (in any medium). But I think it may be possible, though the result is likely to be very different from what we think of as a novel.

The metaphor I like is sculpture. For example, a Calder mobile is a very dynamic thing, and the viewer may participate by pushing one part and observing the resulting forms. Even the act of walking around a sculpture like The Kiss is an essential part of experiencing the work. The viewer has to participate (i.e. interact) to appreciate the artistic expression. So even though sculptures have this participatory aspect, they maintain their artistic integrity. When you look at a Calder or Rodin, there is no question who made the sculpture. And I don’t think Calder felt his expression was compromised by the fact that a viewer (or the wind) could change the sculpture. This is what I was trying to get at in my essay:

The author can control the [combinatoric] explosion by creating only the places and paths-between-places she wants the participant to explore. This is the power of Story Space model. Rather than thinking in terms of a straight line with a beginning and end, the author thinks of a story space through which the participant moves.

In other words, the author should sculpt the story so the reader can walk around it, and move parts around in order to “see” new aspects and gain a deeper sense of the whole. So far, interactive media has been a disappointment to me from an artistic point of view, though I’ve seen some interactive work that impresses me. In part, this is simply because it’s taking longer than I thought for the new medium to develop. But another problem is the computer and computer screen. It’s kind of like watching Citizen Kane through the slots of a zoetrope. All the power and magic is taken out of the experience. Likewise, there’s something wrong with experiencing an artistic expression in the “mouse and crouch” position. I suspect that interactive expression won’t really work until there’s a medium that feeds the experience rather than detracts from it.

The projects were quite evocative, and came at animism from many different directions. Some seemed primarily reactive to interactions, others seemed to be doing their own thing, and yet others were curious in a shy kind of way. A common theme among the projects was the use of fictional back-stories, some quite elaborate, which gave the projects a more full character, suspension of disbelief, and space for imagination.

Reflections on the project, as well as links to details and videos of the interactions below. As a followup to this project, this summer I’ll be leading further research on Animism in a project funded by the Santa Monica based Nokia Research group headed by Sean White.

Through discussion and critique, the class found patterns in these speculative projects that seemed to make the animism work well, summarized in this table.

Student Project Documentation - Click on links for details and videos

Bora Shin’s Curious Mailbox – Watches and emotes about your email	Dee Kim & Chiao Wei Ho’s Future Particle Lab – Scientific research into live nano particles
Link Huang’s Velit – Wishing machine and its minions	Mike Manalo’s Animism: Material Research – Textures and Architecture
Jayne Vidheecharoen’s Botanical Autonoma – Human Spirit Interaction (HSI vs. HCI)	Rubina Ramchandani’s Stonaloid – Traumatized rocks in recovery
Sal Orara’s Tonalis Luminous – A field of tonal, light-sensitive flowers	Joshua McVeigh-Schultz’s Mobile interface devices in their natural state
Aarti Vashisht’a Doodler – A companion that reacts to your writing, and gets bored

This project explores the design opportunities in objects that seem to have inner lives through their expressive behavior.

As Piaget and others have noted, people tend to imagine that inanimate objects are alive – why not leverage this to create more interesting, playful, expressive, desirable, and useful systems?

Creating apparent intentionality for objects offers the potential for fitting more easily into the narrative of daily life, as well as making it easier for people to understand the sometimes complex function and role of digital systems in our lives.

The research would be conducted in summer 2011, in Philip van Allen’s New Ecology of Things Lab (NETLab) at Art Center’s Media Design Program. Professor van Allen will work with Nokia Research staff, as well as engaging select graduate Media Design Program and upper term undergraduate Product Design students to explore the potentials of Object Animism. The project team will create a series of experiments and design speculations that investigate the roles that animism might play, both in how the objects perform their “behind the scenes” tasks, as well as how they express themselves through sound, kinetic motion, and visual displays.

A range of contexts will be selected for the work, such as sensing and reporting, simple affordances for complex tasks, and subtle communication. The project will use and further develop the NETLab Toolkit rapid prototyping system.

An updated, illustrated, and edited version of this post was published in the JohnnyHolland.org magazine about Interaction Design.

http://johnnyholland.org/2011/05/19/the-new-ecology-of-things-slabs-sofducts-and-bespoke-objects/

With the Apple iPad launched and scores of other tablets and e-readers hitting the market, I think it’s important to step back and look at the larger trends. We’re in the middle of a major shift towards ubiquitous computing, cloud based personal storage, and tangible interaction. It’s a shift away from the generic computation typified by the “personal computer,” which never really achieved the individuality or specificity implied by the term “personal.” In short, we’re experiencing the emergence of The New Ecology of Things, where a network of heterogeneous, smart objects and spaces create opportunities for a more personal and meaningful landscape. This is what I’d like to explore:

• Where we’ve been and how the personal computer has made us soulless
• Where we’re about to be #1 with the emergence of digital slabs
• Where we’re about to be #2 with a new form of design that’s a hybrid of software and product
• Where we may be going and the future of the designer in an era of bespoke objects

The Past – The Personal Computer Has Made Us Soulless

There are many signs that all is not well with our day-to-day work life. John Hockenberry’s review of Michael Wolf’s The Transparent City contemplates the crushing homogeneity and conformity of modern work revealed through Wolf’s photographs of life seen through Chicago’s skyscrapers. Describing one photo, Hockenberry sees “12 random floors of eggshell white, computer screens on brown desks, and wall-hung bookshelves.” Sound familiar?

The article goes on to discuss how the environment for “knowledge work” is unlike factories where the space is specifically suited to the activity of making things. The knowledge working context has devolved to the point where “offices have become stacks of boxes for people who get paid to think out of them.” But I believe this is not only a problem of architecture and environmental design. Our daily activity has been squeezed into the narrow channel of interaction with the personal computer and its attendant posture, furniture, and detachment from the needs of the person. The digital tools we use have played a large role in creating this disembodied, deadening uniformity.

Similarly, Matthew B. Crawford has been driven out of the office and into his motorcycle repair shop as described in “Shop Class as Soulcraft: An Inquiry Into the Value of Work.” In the book, Crawford discusses how knowledge work has become vague and disconnected from the concrete, meaningful outcomes of manual labor. But again, perhaps it’s not only the type of work, but the manner in which the work is accomplished. The disconnection from the physical isn’t limited to paper-pushing knowledge workers, but architects, graphic designers, recording engineers and others whom we think of as having “satisfying” jobs where things are made. As creative workers, we’ve seen our day-to-day work compressed from a productive, bodily engaged studio environment down to the almost motionless “mouse-crouch” that plugs us into the virtual. Seduced by the power of the personal computer, we have morphed from active, engaged, social, interactive people to sedentary, soulless slugs perched in front of our glowing screens.

The personal computer has created a homogenous, static and context free environment for work and play that removes activity from the meaningful and productive character of acting and thinking in the embodied, physical environment. This needs to change.

The Emergent – Slabs: A Step Towards Re-Engagement

The iPhone, Android platform, Monome (its clones), Siftables, and now the iPad represent a new form of computing device that I call the slab. Slabs are hand-held, generic platforms with a range of sizes and capabilities (e.g, touch screen, GPS, accelerometer, gyro, WiFi, speaker, mic, etc.) that, in effect, turn into something new with each different application they run.

Slabs are different from personal computers. First, because they have a smaller, simpler form factor and a direct, touch based interaction. Second, because slabs do one thing at a time, in that the device effectively becomes the app once it’s launched, and the separation between software, hardware and interaction dissolves. When you switch apps on a slab, you get a whole new device that engages you as a unified, tangible object. This works because the app is the device (and all the discussions about slabs “multitasking” have new meaning in this context). Third, slabs will eventually be cheap enough that one can use multiple devices at a time, at a location and context appropriate to the physical space and task at hand.

Finally we can begin to disconnect from soulless trap of the (im)personal computer. By using slabs, we customize and re-engage with our environment and feel the consequences of our activity. Instead of being tied to a “workstation,” we can move around the work/play space, utilizing slabs and their spatial relations to one another. We use specialized tools and work practices as slabs morph to the needs of each activity (rather than the other way around). We touch things again. Instead of immersing ourselves in the virtual, we re-engage with people and things in the world. We work more easily with others, sharing and collaborating in physical space, where “here, look” and “take this and work on it” are literal statements that once again become the norm. We customize our tools and ultimately I hope, make or acquire our own specialized tools (see the bespoke objects discussion below).

In addition to their embodied and embedded character, slabs excel at leveraging the affordances of computation and networks. With configuration and data always backed up in the cloud, slabs can be easily reincarnated if scrambled, broken, or lost – and with this, interaction, meaning, and ideas become more important than the object. The cloud also enables different slabs to work across space and time while still being in the here and now. Our digital work is no longer tied to a single workstation, but can manifest in different forms, on different devices, with activity-specific functionality.

In short, there’s the potential to get the best of both worlds – the material and social character of the physical, along with the flexibility, power, and ubiquity of the computational. We gain a heterogeneous collection of devices, specifically suited to the activities at hand. For example, a graphic designer might have a large, work table slab for standing and working on layouts. Or a lap sized slab to sit with and edit an image in a concentrated mood. A narrow slab might sit on a table to keep track of a to-do list. And of course, the designer would use a few 8 1/2″ x 11″ slabs at a client meeting to pass around the table for discussion.

The Emergent – Sofducts: A Challenge for Designers

For the designer of these systems, the slab presents an interesting set of challenges. In particular, there’s a hybrid character to apps. Apps are software, yet as described above, the app becomes something more like a physical product once launched on a slab. This merging of app + slab leads to something I call the software/product or sofduct.

On one hand, the sofduct is distributed like software with almost no cost of goods. On the other hand, the user perceives the sofduct like a product because the interaction feels similar to that of  a manufactured object. You hold it, press buttons, shake it, etc. For example, GPS navigation systems used to be sold as a traditional product, in a box, physically shipped, with a warranty card and customer service phone number. Now, the sofduct version gives you the exact same functionality but is downloaded and runs on a slab as a piece of software. To the user, the end result looks and feels just like the traditional physical product. The sofduct is very disruptive in this way.

For one, whole business models are being destroyed by the sofduct. You can now buy the MotionX-GPS Drive app for $2.99, and get turn-by-turn navigation for $2.99 a month or $25/year (not to mention Google’s free turn-by-turn GPS on Android). In some cases, in-app purchasing of add-ons and features creates a modular “product” model, where the sofduct is actually a range of product possibilities that can be selected and customized by the user. Can traditional GPS units and other physical products survive this kind of competition?

But I’m especially interested in how sofducts disrupt the role of designers. Unlike software for a computer, a sofduct has to meet the expectations for a traditional product. The high-finish aesthetics, ergonomics, and conceptual integrity of physical product design will be assumed by users. Likewise, simplicity and clarity of interaction are critical. The perception of “product-ness” will also influence user expectations for reliability and customer service – we want products to simply work. Because of this, visual and interaction screen designers need to adopt the above considerations and aesthetics of product designers as they develop sofducts. Or better yet, collaborate with a product designer as a member of the design team.

On the other side, product designers entering the sofduct realm need to understand the traditions and expectations for software. Users want constant and rapid, usually free upgrades. Product design tries to get it perfect before launch, since there’s no turning back after you send the device to manufacturing. But with a sofduct, it may be better to put out a really good, but simpler version on the market quickly, and use a software model for product planning where upgrades are rolled out on a strategic schedule. Plus, customization and the integration of media are different from the fixed character of physical products, requiring a deep understanding of interaction, typography, and visual design that requires the experience of software and screen designers.

Sofducts are are new category for design, merging the focus, situated character, and physicality of an object with the malleability, customization, and media richness of software. This requires an integration of disciplines, including software development practices with product design, screen design with haptics, interactive/interaction design with materials sensibility, media production with physical interactions. Further, new business and design opportunities emerge, and require a complete rethinking of design and implementation for this new category.

The Future – Bespoke Objects

As slabs and sofducts create an emerging design landscape today, designers need to prepare for further disruptions and repositioning of their skills. Soon, trends in hardware and software will open up the possibility for low-cost, custom-built systems for individuals and specific applications. In the same way that one can have a bespoke suit tailored to a perfect fit and style, it may soon become possible to have a bespoke object with the hardware, software and design features tailored to the perfect fit and style for you and your intended use.

By this, I don’t mean the custom manufacturing typified by NIKEiD and others in recent years (though that will likely happen as well). What I do mean something literally like the local tailor, working out of a shop around the corner. The production of bespoke objects on the local level is becoming possible because of rapid advances in desktop 3D printing, system-on-a-board components, open-source software and hardware, and the DIY culture growing around these trends.

With cheap, off-the-shelf computational components and the ability to print 3D parts, the digital tailor will soon be able to hang their sign out and make individual or short-run custom objects full of ubicomp goodness. People will want these because a generic, mass-produced slab won’t always be suited to their particular circumstance or activity. Moreover, having a custom designed ensemble of complementary, networked objects, specifically crafted to your way of working will be the hallmark of the enthusiast and expert alike. We’ll want to assemble our own unique ecologies of things, from tiny watch sized objects, through tablets, to big activated interactive walls.

Assuming the bespoke object becomes a reality, what does this mean for the designer and design firms? Will it put designers out of business? I think that in the same way sofducts are disrupting design practices and business models, bespoke objects will create major disruptions for designers. If even a portion of product design and manufacturing moves to a decentralized, local model, many individual designers and design companies will have to adjust.

I see a few possibilities. First, those digital tailors are designers. Or at least the most successful ones will be. Just because some of the parts, software and 3D models will be off-the-shelf, bespoke objects will also have custom aspects and are systems that must be integrated for a specific person or task. That’s the job of a designer. Would it be so bad if designers are small business owners around the corner, selling locally in person and internationally online? Second, the off-the-shelf interactions, interfaces, systems-on-a-board, 3D models, etc. – i.e. the ecosystem around the bespoke object – all need design, and I’d expect a market to develop for small and large organizations to design and produce the necessary (virtual and physical) components that enable the digital tailor to operate.

The New Ecology of Things

Nearly 20 years ago, Mark Weiser published his seminal paper on ubiquitous computing in Scientific American, “The Computer for the 21st Century” (scan, reprint in PDF – you really should read it!). This remarkably prescient work predicts much of what’s becoming a reality today (and perhaps Apple’s iPad name is a tip-of-the-hat to Weiser’s taxonomy of tabs, pads, and boards). More recently, Microsoft showed their vision of 2019 video. It’s a great visualization of the potential fluidity of interaction, but the homogeneity and sense of virtuality (a window into something) does not capture the more tangible, gritty, idiosyncratic, embodied, embedded character I hope for in The New Ecology of Things.

Both visions emphasize the screen as the dominant system. Certainly the screen’s power to change dynamically; show combinations of text, image, animation, and video; and to support interaction through touch is remarkable. Yet I think one of the most interesting challenges for designers is to look beyond the screen slab, and imagine how other computationally enhanced objects with texture, kinetic motion, haptic feedback, sound, and light can be integrated into this new ecology of things. Let’s make sure we leverage the joy and power of all the human senses and abilities in our designs.

And the doubters are right, for those of us who want a cool, small, OS X laptop thing, the iPad does not hit the mark (of course, as the iPad matures it will get many of these things – the inevitable 4.0 release of the iPhone OS will likely address the multitasking and app organization issues. And maybe Adobe and Apple will finally make nice for the Flash plug-in. And the 2011 version will have a camera etc.).

But people who want a better laptop aren’t the target audience for the iPad. Just like people who wanted a better Blackberry weren’t the target audience for the original iPhone. And people who wanted a green-washed sports-car were not the target audience for the Prius. Guess what? These products were a success anyway, because they met a new need and found a new audience.

So what is the iPad? Well, that remains to be seen. It will evolve in the next year as developers turn the iPad into a range of completely new things that, once they exist, will be essential for many people. To start all this off, Apple gives us the basic foundation and a decent value proposition:

• Read rich media books/magazines/newspapers
• Comfortably browse the web
• Use interactive textbooks and other learning material
• Work with personal media (photos, music, video)
• Play games
• Shift low-intensity computer stuff off the computer (email, todo lists, calendars, presentations, note taking, etc.)
• And as an add-on value, provide an admittedly compromised level of computer substitution for word processing, spreadsheets, etc. so we don’t always have to drag around a laptop.

That alone is a pretty good deal for $500. But what will come, and will likely make the iPad a major success, is a range of new apps that turn the iPad into an incredible device for doing more specific kinds of activities. For example, recording a song – it can be a complete recording system and tangible interface with faders, knobs, transport controls, etc. Or as remote control for your house and entertainment system. Or as a painting canvas. Or as a device that sits next to, and is an adjunct to your computer - wacom tablet, todo list, email, application switcher, etc. Or to organize your genealogy. Or to do scrapbooking (really, this could sell a millions units alone). Or plan a trip. Or evaluate X-Rays at your patient’s bedside… You get the idea.

I call these devices Slabs. The iPad, iPhone, Andriod, etc. are generic platforms that, via an app, turn into a product. And a 10″ Slab with multi-touch surface can be a lot of different products.

More to come in the following days on my thoughts about SLABS, SOFTDUCTS, and BESPOKE OBJECTS.

Each of these play a role in people’s lives with broad effects in terms of activities, emotions, thinking, tactility, social interactions, creativity, work, play, and more. Even the “functional” hammer does more than solve the problem of putting nails into wood – it feels right in the hand, it gains a patina over time that makes it personal, in a pinch it will open a beer bottle, and you can use it to repair a church after Katrina.

In particular, if we think about Interactive Design, the highest goal should be to empower people to create their own meaning spaces, not solve pre-determined problems or even make great experiences. As I’ve discussed in my Productive Interaction paper and in The New Ecology of Things, design plays a greater role than serving tasks and solving problems. The things in our lives communicate, create social exchanges, and enable us to manipulate both the tangible and the idea. They afford creative abuse and invention. Forget solving problems, design things to be productive, embodied, mythological, meaningful.

In other words, by tracking what you and others do and create (emails, blog entries, tweets, browsing activity, shopping, etc.), and by scouring the web and analyzing its content, these systems make sense of the web in a much more sophisticated way than the brute force kind of searching that Google does. So it could find correlations, generate connections, optimize searches, make you aware of implicit networks of interest, and generally act on your behalf to both filter the incoming avalanche of data, and provide better/faster means to get to interesting information that you might not otherwise find.

While this idea is related to the kinds of recommendations that Amazon and other sites do, it is stronger because it aggregates a lot more activity and content beyond the silo of a single site. Plus, the ultimate expression of the implicit web (I hope) is that the user will have more control, and can “dial-in” the criteria of a search or automated task to their specific interests at that moment, rather than being stuck with some company’s idea of your interests. This idea relates to my essay on Productive Interaction, where the design of these systems is not about creating enveloping, persuasive experiences (as experience design dictates), but designing contexts where users are empowered to create their own meaning spaces.

Related LINKS below

Some posts by an investor in implicit web companies:
http://www.feld.com/wp/archives/2009/04/the-maturing-of-the-implicit-web.html
http://www.foundrygroup.com/blog/archives/2008/03/theme-implicit-web.php

ReadWriteWeb’s take on it:
http://www.readwriteweb.com/archives/the_implicit_web_lastfm_amazon_google.php

More sophisticated, semantic based tagging

http://www.readwriteweb.com/archives/common_tag_brings_standards_to_metadata.php

Some companies doing it:
http://getglue.com/brief.php
http://www.oneriot.com/
http://www.lijit.com/
http://www.orch8.net/

From the Feld.com post:

We think of the technologies that fall under the implicit web theme as a next-generation set of applications, tools and infrastructure that stitch together a long list of interrelated and overlapping ideas: the academic and theoretical ideas behind the Semantic Web, the utility of social networks and social media, crowd sourcing/wisdom-of-crowds, folksonomy, user attention data, advanced search and content analysis tools, lifestream analysis and numerous others.

When combined, these technologies offer the promise of a more unified computing environment that spans the applications where a user consumes and creates information (email clients, web browsers, RSS readers, etc) and is aware of the user’s preferences, interests and interpersonal relationships without requiring a ton of heavy lifting on the user’s part to get useful work done.

All projects are working interactive demos that use the Make Controller in combination with our NET Lab Toolkit (Pen & Book didn’t use the Make).

Net Desk – Chris Becker & Ryan D’orazi
Focuses on the affordances of paper comps of magazine and other printed material. Each piece of paper is tagged, and acts as a vector into all of its versions, related documents, and relationships with the people and files that created the content of the page. By swiping the paper over different areas of the desk, the user instantly sees these different aspects of the page on a large screen. Reflects the activities of real-world magazine production. Paper, custom built table, Barcode reader, Wii controller, 50″ display, light sensors.

Whisper Stones – Kylan Coats
Plays with the idea mythical character of the stone by allowing people to lean down, pick up a one of several stones, and anonymously leave or hear and audio message. Small and large stones, $10 radio shack solid state recorders, switches.

Pen and Book – Chris Lauritzen & Hunter Sebresos
Explores and embraces the affordances of pen and paper by imagining simple, non-precious, mass produced and cheap digital pens and e-books. Enables the user to write on standard paper, yet instantly email the new document. Also, mark up a book and access the notes with any reader, anywhere. Paper, modified ball-point pen, Wacom tablets, projector, Flash, AppleScript.

Memory Apparatus – Hyun Yang
Inverts the “problem” of ubiquitous surveillance video, and demonstrates how people can access the histories of their own lives through the near future of continuous video capture of everything, everywhere. By rotating the two axis of the Astrolabe like object, one can dial in moments and locations of your life, seeing yourself from the perspective of a surveillance camera. Knob, Wii controller, video.

Postgeheimnis (The mail project) – Christiane Holzheid & Julia Tsao
A series of speculations (some mocked up in video scenarios, others prototypes) that explores ideas for applying some of the beneficial affordances of traditional mail and packages to electronic forms of mail. For example, in one of the working prototypes, an authorized friend is able to leave a video message for you and assign it to an object (e.g. a lamp) in your home. The lamp blinks when a message is waiting, and you simply touch the lamp see the video message played back in place of the light cast by the lamp. Micro projector, speaker, pressure sensors, two computers, Flash media server.

Shop Conscious – Nicole Chan & Haelim Paek
A system that enables the shopper to rate and find more information as they shop. They can create different rating profiles for themselves, weighting sustainability, product quality, and social network criteria. It includes a portable and dressing room media component. iPhone with LiveView app streaming flash from laptop, proximity sensors, light sensor.

Brainstorming Egg – Haemi Yoon
This project imagines a quarter sized projector that can be stuck to any surface (like your pant-leg) and turns it into a brainstorming area. Using a egg shaped object as the interface, the user can gesture to create a topic, manipulate media (e.g. scrub video), and make connections between ideas. Accelerometer, XBee wireless module, Wii controller in combination with Jonny Lee’s whiteboard software.

NetCreator – Austin Lee & Yoo Kyoung Noh
By imagining a future startup company, this project uses a series of video simulations and working prototypes to explore how everyday objects can mediate communications and create new meanings in their surroundings. For example, the working prototype demonstrated how a user could read information on their coffee cup, transfer that article to another person’s cup, send secret messages, or seamlessly transition from the short version on the cup to a full version displayed on an e-paper table. iPod Touch with LiveView app streaming flash from laptop, pressure sensor, projector, knob.

I think the way to go is to cast it in terms of designing behavior.  There are many principles and concepts of designing interesting, rich, meaningful behavior that I think could be developed, some of which is instantiated in code, other aspects in the mechanical design (the turning of a doorknob or the page of a book for example), and others in the conceptual design.  This shift to behavior design as an overarching concept that encompasses computation may make it more interesting and relevant to designers.

I’m not arguing that interaction should be cinematic, as Steven Spielberg recently suggested to students at USC’s program in gaming. Film is one medium, interaction is another with it’s own grammar, character, and means of communication. What I am suggesting is that interaction designers should provide users with the sense of magic, power, and real-time manipulation implied by the bullet-time technique.