Image Image Image Image Image
“Design is not just what it looks like and feels like. Design is how it works.”

– Steve Jobs

Scroll to Top

To Top

Featured Projects

Delft AI Toolkit

On 16, Jan 2018 | In | By phil

Delft AI Toolkit

A tool for the design of AI

We are entering an age of pervasive machine intelligence, and there is an urgent need to develop strategies for the design of these new ecologies. The Delft AI Toolkit (GitHub repo) is a platform to easily create working AI prototypes that intelligently behave – see, hear, process, manipulate data, move, and speak.

Work in progress videos at the bottom of the post

The goal is to empower designers to go beyond the cliches of AI, invent new interactions, and bring smart collaborative systems into people’s lives that are diverse, humane, personal, and inspiring.

This project builds on my prior research (NETLab Toolkit and Animistic Design), and will ultimately allow designers to easily and iteratively shape the perceived personality, character, and behavior of an AI product. 

The tool provides a drag-and-drop, visual programming environment to build the interaction and behavior of an AI system. In addition to building the logic of the application, the tool allows the designer to incrementally prototype the AI, form and behavior. First by simulating the AI, before committing to the training and implementation of an actual machine learning algorithm. Second, by simulating the physical form and behavior using augmented reality, before committing to hardware that will be in the final device.

Simulate, then implement AI

To support the simulation of the AI, the design tool will have a â€œpersonality control panel” to shape the system’s behavior and data sources. It will also have a visual marionette system so the designer can “wizard-of-oz” AI behaviors in real time while observing users. The tool will then allow the implementation of these approaches to AI in a functional machine learning system.

Simulate, then implement hardware

We’ll also be exploring the potentials of using augmented reality as part of the designer’s process. Creating working, physical prototypes is critical to the design process, but it can be time consuming and expensive.

The tool will support augmented reality simulations of the AI project, so a designer can quickly test and update the form/behavior of an embodied experience without the complexities of electronics and mechanics. The designer or user will “see” and interact with the AI system as an object in real space through AR. Then the tool can deploy the same simulated virtual behaviors into a physical prototype.

Delft Toolkit Architecture

Videos of work in progress

Version 2, voice and object recognition

Basics of using the system

Designing an interaction with embodied sensing

Training machine learning of gestures

Recognizing speech and objects

Stay Up to Date

Read more about my ideas and this project in Design United’s interview. And follow the project on my process blog: