Object & Screen Speculations

Update, September 2017: I recently watched this after many years, because I randomly came across a SlideShare from a workshop that David Sherwin and Aaron Rincover ran back in 2010. The workshop was called “Prototyping Interaction with Video Scenarios,” and it references my video as an example of rough prototyping and interaction (slide 16).

When I made the video I was thinking about how to use tangible objects as a way to interact with screens. But what struck me watching it in 2017 is how relevant this kind of interaction is for AR and VR.

I’m especially focused on AR, and the potential for doing creative and analytic work with virtual, AI enhanced, morphing things in the AR environment. The current AR/VR controllers are a start, but they are pretty crude. Having nuanced, physical objects to work with virtual objects seems like a very rich area to explore. Some of my recent work in AR is here: Animistic Collaborators in Mixed Reality

The funny thing is that finding this video useful ten years later only proves the point that David and Aaron were making. Using low-fidelity videos can be so valuable, because they allow one to prototype a vision of interaction that’s open-ended enough to contribute to the design process, not simply be an artifact of the design.

Original description, January 2008

I’m very interested in how tangible objects can be used in interesting ways to interact with information on screens. This video collects together a series of experiments on the use of a range of object prototypes. In making these, I imagined a screen in front of me (in some cases a standard size screen, in other cases a wall sized screen), and manipulated the various objects as if I was controlling and interacting with content on the screen. It was more an experiment in the affordance of the objects in relation to screens than thinking of specific applications.

This project was conducted in 2007, with the help of my graduate students Jonathan Jarvis, and Parker Kuncl.