An updated, illustrated, and edited version of this post was published in the JohnnyHolland.org magazine about Interaction Design.

http://johnnyholland.org/2011/05/19/the-new-ecology-of-things-slabs-sofducts-and-bespoke-objects/

With the Apple iPad launched and scores of other tablets and e-readers hitting the market, I think it’s important to step back and look at the larger trends. We’re in the middle of a major shift towards ubiquitous computing, cloud based personal storage, and tangible interaction. It’s a shift away from the generic computation typified by the “personal computer,” which never really achieved the individuality or specificity implied by the term “personal.” In short, we’re experiencing the emergence of The New Ecology of Things, where a network of heterogeneous, smart objects and spaces create opportunities for a more personal and meaningful landscape. This is what I’d like to explore:

• Where we’ve been and how the personal computer has made us soulless
• Where we’re about to be #1 with the emergence of digital slabs
• Where we’re about to be #2 with a new form of design that’s a hybrid of software and product
• Where we may be going and the future of the designer in an era of bespoke objects

The Past – The Personal Computer Has Made Us Soulless

There are many signs that all is not well with our day-to-day work life. John Hockenberry’s review of Michael Wolf’s The Transparent City contemplates the crushing homogeneity and conformity of modern work revealed through Wolf’s photographs of life seen through Chicago’s skyscrapers. Describing one photo, Hockenberry sees “12 random floors of eggshell white, computer screens on brown desks, and wall-hung bookshelves.” Sound familiar?

The article goes on to discuss how the environment for “knowledge work” is unlike factories where the space is specifically suited to the activity of making things. The knowledge working context has devolved to the point where “offices have become stacks of boxes for people who get paid to think out of them.” But I believe this is not only a problem of architecture and environmental design. Our daily activity has been squeezed into the narrow channel of interaction with the personal computer and its attendant posture, furniture, and detachment from the needs of the person. The digital tools we use have played a large role in creating this disembodied, deadening uniformity.

Similarly, Matthew B. Crawford has been driven out of the office and into his motorcycle repair shop as described in “Shop Class as Soulcraft: An Inquiry Into the Value of Work.” In the book, Crawford discusses how knowledge work has become vague and disconnected from the concrete, meaningful outcomes of manual labor. But again, perhaps it’s not only the type of work, but the manner in which the work is accomplished. The disconnection from the physical isn’t limited to paper-pushing knowledge workers, but architects, graphic designers, recording engineers and others whom we think of as having “satisfying” jobs where things are made. As creative workers, we’ve seen our day-to-day work compressed from a productive, bodily engaged studio environment down to the almost motionless “mouse-crouch” that plugs us into the virtual. Seduced by the power of the personal computer, we have morphed from active, engaged, social, interactive people to sedentary, soulless slugs perched in front of our glowing screens.

The personal computer has created a homogenous, static and context free environment for work and play that removes activity from the meaningful and productive character of acting and thinking in the embodied, physical environment. This needs to change.

The Emergent – Slabs: A Step Towards Re-Engagement

The iPhone, Android platform, Monome (its clones), Siftables, and now the iPad represent a new form of computing device that I call the slab. Slabs are hand-held, generic platforms with a range of sizes and capabilities (e.g, touch screen, GPS, accelerometer, gyro, WiFi, speaker, mic, etc.) that, in effect, turn into something new with each different application they run.

Slabs are different from personal computers. First, because they have a smaller, simpler form factor and a direct, touch based interaction. Second, because slabs do one thing at a time, in that the device effectively becomes the app once it’s launched, and the separation between software, hardware and interaction dissolves. When you switch apps on a slab, you get a whole new device that engages you as a unified, tangible object. This works because the app is the device (and all the discussions about slabs “multitasking” have new meaning in this context). Third, slabs will eventually be cheap enough that one can use multiple devices at a time, at a location and context appropriate to the physical space and task at hand.

Finally we can begin to disconnect from soulless trap of the (im)personal computer. By using slabs, we customize and re-engage with our environment and feel the consequences of our activity. Instead of being tied to a “workstation,” we can move around the work/play space, utilizing slabs and their spatial relations to one another. We use specialized tools and work practices as slabs morph to the needs of each activity (rather than the other way around). We touch things again. Instead of immersing ourselves in the virtual, we re-engage with people and things in the world. We work more easily with others, sharing and collaborating in physical space, where “here, look” and “take this and work on it” are literal statements that once again become the norm. We customize our tools and ultimately I hope, make or acquire our own specialized tools (see the bespoke objects discussion below).

In addition to their embodied and embedded character, slabs excel at leveraging the affordances of computation and networks. With configuration and data always backed up in the cloud, slabs can be easily reincarnated if scrambled, broken, or lost – and with this, interaction, meaning, and ideas become more important than the object. The cloud also enables different slabs to work across space and time while still being in the here and now. Our digital work is no longer tied to a single workstation, but can manifest in different forms, on different devices, with activity-specific functionality.

In short, there’s the potential to get the best of both worlds – the material and social character of the physical, along with the flexibility, power, and ubiquity of the computational. We gain a heterogeneous collection of devices, specifically suited to the activities at hand. For example, a graphic designer might have a large, work table slab for standing and working on layouts. Or a lap sized slab to sit with and edit an image in a concentrated mood. A narrow slab might sit on a table to keep track of a to-do list. And of course, the designer would use a few 8 1/2″ x 11″ slabs at a client meeting to pass around the table for discussion.

The Emergent – Sofducts: A Challenge for Designers

For the designer of these systems, the slab presents an interesting set of challenges. In particular, there’s a hybrid character to apps. Apps are software, yet as described above, the app becomes something more like a physical product once launched on a slab. This merging of app + slab leads to something I call the software/product or sofduct.

On one hand, the sofduct is distributed like software with almost no cost of goods. On the other hand, the user perceives the sofduct like a product because the interaction feels similar to that of  a manufactured object. You hold it, press buttons, shake it, etc. For example, GPS navigation systems used to be sold as a traditional product, in a box, physically shipped, with a warranty card and customer service phone number. Now, the sofduct version gives you the exact same functionality but is downloaded and runs on a slab as a piece of software. To the user, the end result looks and feels just like the traditional physical product. The sofduct is very disruptive in this way.

For one, whole business models are being destroyed by the sofduct. You can now buy the MotionX-GPS Drive app for $2.99, and get turn-by-turn navigation for $2.99 a month or $25/year (not to mention Google’s free turn-by-turn GPS on Android). In some cases, in-app purchasing of add-ons and features creates a modular “product” model, where the sofduct is actually a range of product possibilities that can be selected and customized by the user. Can traditional GPS units and other physical products survive this kind of competition?

But I’m especially interested in how sofducts disrupt the role of designers. Unlike software for a computer, a sofduct has to meet the expectations for a traditional product. The high-finish aesthetics, ergonomics, and conceptual integrity of physical product design will be assumed by users. Likewise, simplicity and clarity of interaction are critical. The perception of “product-ness” will also influence user expectations for reliability and customer service – we want products to simply work. Because of this, visual and interaction screen designers need to adopt the above considerations and aesthetics of product designers as they develop sofducts. Or better yet, collaborate with a product designer as a member of the design team.

On the other side, product designers entering the sofduct realm need to understand the traditions and expectations for software. Users want constant and rapid, usually free upgrades. Product design tries to get it perfect before launch, since there’s no turning back after you send the device to manufacturing. But with a sofduct, it may be better to put out a really good, but simpler version on the market quickly, and use a software model for product planning where upgrades are rolled out on a strategic schedule. Plus, customization and the integration of media are different from the fixed character of physical products, requiring a deep understanding of interaction, typography, and visual design that requires the experience of software and screen designers.

Sofducts are are new category for design, merging the focus, situated character, and physicality of an object with the malleability, customization, and media richness of software. This requires an integration of disciplines, including software development practices with product design, screen design with haptics, interactive/interaction design with materials sensibility, media production with physical interactions. Further, new business and design opportunities emerge, and require a complete rethinking of design and implementation for this new category.

The Future – Bespoke Objects

As slabs and sofducts create an emerging design landscape today, designers need to prepare for further disruptions and repositioning of their skills. Soon, trends in hardware and software will open up the possibility for low-cost, custom-built systems for individuals and specific applications. In the same way that one can have a bespoke suit tailored to a perfect fit and style, it may soon become possible to have a bespoke object with the hardware, software and design features tailored to the perfect fit and style for you and your intended use.

By this, I don’t mean the custom manufacturing typified by NIKEiD and others in recent years (though that will likely happen as well). What I do mean something literally like the local tailor, working out of a shop around the corner. The production of bespoke objects on the local level is becoming possible because of rapid advances in desktop 3D printing, system-on-a-board components, open-source software and hardware, and the DIY culture growing around these trends.

With cheap, off-the-shelf computational components and the ability to print 3D parts, the digital tailor will soon be able to hang their sign out and make individual or short-run custom objects full of ubicomp goodness. People will want these because a generic, mass-produced slab won’t always be suited to their particular circumstance or activity. Moreover, having a custom designed ensemble of complementary, networked objects, specifically crafted to your way of working will be the hallmark of the enthusiast and expert alike. We’ll want to assemble our own unique ecologies of things, from tiny watch sized objects, through tablets, to big activated interactive walls.

Assuming the bespoke object becomes a reality, what does this mean for the designer and design firms? Will it put designers out of business? I think that in the same way sofducts are disrupting design practices and business models, bespoke objects will create major disruptions for designers. If even a portion of product design and manufacturing moves to a decentralized, local model, many individual designers and design companies will have to adjust.

I see a few possibilities. First, those digital tailors are designers. Or at least the most successful ones will be. Just because some of the parts, software and 3D models will be off-the-shelf, bespoke objects will also have custom aspects and are systems that must be integrated for a specific person or task. That’s the job of a designer. Would it be so bad if designers are small business owners around the corner, selling locally in person and internationally online? Second, the off-the-shelf interactions, interfaces, systems-on-a-board, 3D models, etc. – i.e. the ecosystem around the bespoke object – all need design, and I’d expect a market to develop for small and large organizations to design and produce the necessary (virtual and physical) components that enable the digital tailor to operate.

The New Ecology of Things

Nearly 20 years ago, Mark Weiser published his seminal paper on ubiquitous computing in Scientific American, “The Computer for the 21st Century” (scan, reprint in PDF – you really should read it!). This remarkably prescient work predicts much of what’s becoming a reality today (and perhaps Apple’s iPad name is a tip-of-the-hat to Weiser’s taxonomy of tabs, pads, and boards). More recently, Microsoft showed their vision of 2019 video. It’s a great visualization of the potential fluidity of interaction, but the homogeneity and sense of virtuality (a window into something) does not capture the more tangible, gritty, idiosyncratic, embodied, embedded character I hope for in The New Ecology of Things.

Both visions emphasize the screen as the dominant system. Certainly the screen’s power to change dynamically; show combinations of text, image, animation, and video; and to support interaction through touch is remarkable. Yet I think one of the most interesting challenges for designers is to look beyond the screen slab, and imagine how other computationally enhanced objects with texture, kinetic motion, haptic feedback, sound, and light can be integrated into this new ecology of things. Let’s make sure we leverage the joy and power of all the human senses and abilities in our designs.

All projects are working interactive demos that use the Make Controller in combination with our NET Lab Toolkit (Pen & Book didn’t use the Make).

Net Desk – Chris Becker & Ryan D’orazi
Focuses on the affordances of paper comps of magazine and other printed material. Each piece of paper is tagged, and acts as a vector into all of its versions, related documents, and relationships with the people and files that created the content of the page. By swiping the paper over different areas of the desk, the user instantly sees these different aspects of the page on a large screen. Reflects the activities of real-world magazine production. Paper, custom built table, Barcode reader, Wii controller, 50″ display, light sensors.

Whisper Stones – Kylan Coats
Plays with the idea mythical character of the stone by allowing people to lean down, pick up a one of several stones, and anonymously leave or hear and audio message. Small and large stones, $10 radio shack solid state recorders, switches.

Pen and Book – Chris Lauritzen & Hunter Sebresos
Explores and embraces the affordances of pen and paper by imagining simple, non-precious, mass produced and cheap digital pens and e-books. Enables the user to write on standard paper, yet instantly email the new document. Also, mark up a book and access the notes with any reader, anywhere. Paper, modified ball-point pen, Wacom tablets, projector, Flash, AppleScript.

Memory Apparatus – Hyun Yang
Inverts the “problem” of ubiquitous surveillance video, and demonstrates how people can access the histories of their own lives through the near future of continuous video capture of everything, everywhere. By rotating the two axis of the Astrolabe like object, one can dial in moments and locations of your life, seeing yourself from the perspective of a surveillance camera. Knob, Wii controller, video.

Postgeheimnis (The mail project) – Christiane Holzheid & Julia Tsao
A series of speculations (some mocked up in video scenarios, others prototypes) that explores ideas for applying some of the beneficial affordances of traditional mail and packages to electronic forms of mail. For example, in one of the working prototypes, an authorized friend is able to leave a video message for you and assign it to an object (e.g. a lamp) in your home. The lamp blinks when a message is waiting, and you simply touch the lamp see the video message played back in place of the light cast by the lamp. Micro projector, speaker, pressure sensors, two computers, Flash media server.

Shop Conscious – Nicole Chan & Haelim Paek
A system that enables the shopper to rate and find more information as they shop. They can create different rating profiles for themselves, weighting sustainability, product quality, and social network criteria. It includes a portable and dressing room media component. iPhone with LiveView app streaming flash from laptop, proximity sensors, light sensor.

Brainstorming Egg – Haemi Yoon
This project imagines a quarter sized projector that can be stuck to any surface (like your pant-leg) and turns it into a brainstorming area. Using a egg shaped object as the interface, the user can gesture to create a topic, manipulate media (e.g. scrub video), and make connections between ideas. Accelerometer, XBee wireless module, Wii controller in combination with Jonny Lee’s whiteboard software.

NetCreator – Austin Lee & Yoo Kyoung Noh
By imagining a future startup company, this project uses a series of video simulations and working prototypes to explore how everyday objects can mediate communications and create new meanings in their surroundings. For example, the working prototype demonstrated how a user could read information on their coffee cup, transfer that article to another person’s cup, send secret messages, or seamlessly transition from the short version on the cup to a full version displayed on an e-paper table. iPod Touch with LiveView app streaming flash from laptop, pressure sensor, projector, knob.

I’ll be speaking about The New Ecology of Things and our NET Lab tools at the flashbelt conference that runs from June 8th to June 11th, 2008 in Minneapolis, MN. This conference focuses on the in-depth issues of designing and developing real interactive applications. Sessions range from experience design from Motion Theory‘s perspective, to animation design, sound design, developing in Adobe’s AIR, programming in processing, physical computing, to working with the Papervision3D library in Flash.

On April 25th 2008, Anne Burdick (MDP Department Chair), Nik Hafermaas (Dean of Communication Design @ Art Center) and I gave a talk at the USC Interactive Media Arts and Practice Program to discuss the MDP’s New Ecology of Things research initiative. This talk was webcast, and the web recording of it can be seen on Adobe’s education site.