Featured Projects
On 16, Jan 2018 | In Featured Speculative Tangible Teaching | By phil
Delft AI Toolkit
A tool for the design of AI
We are entering an age of pervasive machine intelligence, and there is an urgent need to develop strategies for the design of these new ecologies. The Delft AI Toolkit (GitHub repo) is a platform to easily create working AI prototypes that intelligently behave – see, hear, process, manipulate data, move, and speak.
Work in progress videos at the bottom of the post
The goal is to empower designers to go beyond the cliches of AI, invent new interactions, and bring smart collaborative systems into people’s lives that are diverse, humane, personal, and inspiring.
This project builds on my prior research (NETLab Toolkit and Animistic Design), and will ultimately allow designers to easily and iteratively shape the perceived personality, character, and behavior of an AI product.
The tool provides a drag-and-drop, visual programming environment to build the interaction and behavior of an AI system. In addition to building the logic of the application, the tool allows the designer to incrementally prototype the AI, form and behavior. First by simulating the AI, before committing to the training and implementation of an actual machine learning algorithm. Second, by simulating the physical form and behavior using augmented reality, before committing to hardware that will be in the final device.
Simulate, then implement AI
To support the simulation of the AI, the design tool will have a “personality control panel†to shape the system’s behavior and data sources. It will also have a visual marionette system so the designer can “wizard-of-oz†AI behaviors in real time while observing users. The tool will then allow the implementation of these approaches to AI in a functional machine learning system.
Simulate, then implement hardware
We’ll also be exploring the potentials of using augmented reality as part of the designer’s process. Creating working, physical prototypes is critical to the design process, but it can be time consuming and expensive.
The tool will support augmented reality simulations of the AI project, so a designer can quickly test and update the form/behavior of an embodied experience without the complexities of electronics and mechanics. The designer or user will “see†and interact with the AI system as an object in real space through AR. Then the tool can deploy the same simulated virtual behaviors into a physical prototype.

Delft Toolkit Architecture
Videos of work in progress
Version 2, voice and object recognition
Designing an interaction with embodied sensing
Training machine learning of gestures
Recognizing speech and objects
Stay Up to Date
Read more about my ideas and this project in Design United’s interview. And follow the project on my process blog: ai-toolkit.tumblr.com