KinaTrons: Puppeteering & Play

(2022)
Role: Inventor, Designer, Hardware, Software

An experiment in puppeteering and play, KinaTrons is a reconfigurable, kinetic toy that responds to your body movements. The P5.js-based web app uses PoseNet to track your skeleton, and then communicates various body angles to Arduino via P5.serialcontrol.

Above is the original concept storyboard, showing various pieces being assembled and interacted with. Below, you can see recent renders of the assembled components and web app.

There are three major components to this project. The first being a web based user interface, built in P5.JS. The Kinatrons app uses the PoseNet library to estimate various body points from your video camera feed. Then the app calculates body angles from those points, and send that data in a string to the toy via serial connection. The data is received by the toy via The Torso module, which contains an Arduino and custom PCB that allows me to connect 6 servo-driven joints at once. The electronics within the Torso drive these servos, and makes sure that the joint angles are quickly translated from video feed to actual toy movement. The final component of Kinatrons is of course those servo driven joints, which can be assembled in various configurations to animate the toy in different ways.