When we were asked by our friends at Davies Toews Architecture to collaborate on something for there gallery and project space Rabbit Rabbit, we thought it would be a fun opportunity to continue our explorations in blurring the lines between interactive digital environments and physical space, materials, and objects. We had access to four standard Epson projectors so thought it would be an interesting exercise to map three walls of the space with interactive content. The space was occupied with various exhibition elements and offcuts made for previous shows. The idea was simple enough, to arrange those elements along the walls and have the digital content interact with the “terrain” made of physical elements. The model above was a quick test of most of the elements we would need to create the installation.

The Processing sketch below drove the interactive content. We wanted to keep the interactive content quirky and fun, but paired down so the motion and physics became the main focus of how the digital objects (circles and squares) interacted with the physical elements. We used the Fisica Processing library for the physics. One of the biggest challenges was outputting the visuals to the four projectors. We used Touch Designer to split and blind the digital content, but we needed to be able to display the content in processing much smaller than the aspect ratio of the 4 projectors combined, 5120 x 800. We ended up running the physics simulation in the background of processing at that resolution and then queried the positions, rotation, and color of each shape to redraw it at a smaller scale in processing. We also sent this data to Touch Designer via OSC and redrew the physics simulation there. That gave us the most control over everything. You can download the source files for the processing sketch exported as a Windows application here: Bounce.zip

The sketch above also allowed us to adjust the silhouette of the physical objects along the walls so it would appear as if the digital shapes are colliding with them. We started with the composite image below and used OpenCV to create a contour of the shapes. We then saved that as a *.txt file from processing that we could continue to edit on site and save.

We started by sketching out the space and the placement of the projectors. We also did tests in the studio with the initial Processing sketch and sending OSC to Touch Designer, dividing the redrawn shapes on 4 monitors to make sure there was no real drop in frame rate and that the OSC stream was consistent.

Our initial tests of mapping and blending the 4 projectors up in the space with a grid.

To further blur the boundaries between digital and physical space we wanted to allow visitors to interact with the installation in a tactile way. We decided to use arcade buttons as a way to allow people to add and delete shapes. We used arcade buttons from Adafruit and two of their arcade button breakouts to link 6 buttons together. Adafruit is not only a great resource for fun interactive hardware but they also provide such good documentation and guides for so many of their products. We used their breakout guide to quickly get everything up and running with Arduino. We used an Arduino Uno to send the signals from the buttons and breakout to processing via serial port.

Below is a video of the final installation:

We also used a physics simulation for the title graphics for the installation.