When Moogfest invited us to design an interactive installation for its annual festival, a convergence of artistry and engineering in the spirit of Bob Moog, pioneer of the analog synthesizer, we had a completely blank slate to work from. For us, this is somewhat abnormal. In our usual design process, we lay out clear constraints and a specific goal for our clients. This time, there were no limits to how creative we could be, and figuring out where to start was one of the the hardest parts of the project. Here’s how we went from a vague idea about creativity and music to a gestural instrument composed of six giant inflatable beach balls for collaborative music making: a beach ball synth!
We knew that we wanted to channel the spirit of Bob Moog in our installation, so we began with a brainstorming and prototyping phase where we focused on bridging music and technology with creativity. We really wanted to break the barrier that prevents people from expressing themselves musically, and we also wanted to capture something of the Moogfest zeitgeist. Two ideas emerged—one around music, and one around narrative and synthesis based on the statements of the people at the festival. We loved both and wanted them to coalesce, but we were concerned they might be oil and water.
To find out, we began prototyping. We made a quadraphonic phase-music vocoder sampler in hopes of making spoken phrases musical; a computer-vision Post-it® Note step sequencer to bring musical life to an unintimidating object; and generative word games to try to tease out a crowd-sourced narrative arc of the festival. But nothing felt quite right, so we went back to the drawing board and asked ourselves, what is it that we want people to feel?
A Step Back
To get more perspective, we looked for reference points. We found inspiration in large scale installations from artists like Zimoun, where a simple idea like the sound of a cotton ball hitting a cardboard box took on new effect and meaning when scaled. It prompted us to wonder, what if we scaled the acoustic sound of balloons? What are the sounds the balloons should make, where should the sound come from, and how might that capture or express the narrative of the festival?
Quickly and tragically, we discovered that combining a bunch of large balloons with an industrial fan was not quite as mind blowing as we had expected. But, still, their innocuous beauty made balloons seem like a magical invitation to play.
We sought out people who had worked on similar projects, and who were not steeped in our brainstorming assumptions. We talked to Jack Gale from CCA, who had created an interactive balloon project, invited a group from the Exploratorium to brainstorm and test out a rough prototype of balloon word games, and worked to find balloons that were big enough and robust enough for us to embed light elements and sensors in them.
Originally, we had based our design around an 8 foot by 16 foot box. Instead, we chose a covered outdoor basketball court. Huge! Central! Beautiful!
The scale was drastically larger and the lights we had planned to embed couldn’t compete with daylight. To make it work, we scaled the beach ball diameter from 2.5 feet to 8 feet to better fit the space (we had already swapped the balloons for beach balls, since the materials are more robust), eliminated light elements, and put the emphasis on sound derived from beach ball interaction. Since we were at a music festival, it was poetic for sound to be the primary feedback mechanism.
Our friends at the Exploratorium invited us to do a trial install at their museum, which quickly brought the interaction questions to the forefront. You learn a lot quickly when the public kicks the tires. When people walked up to the beach balls, their first instinct was to hit them. But we didn’t want the beach balls to just work like a button—we wanted the installation to be more nuanced, and produce different sounds based on where and how each ball was hit. To do it, we used the nine degree of freedom inertial motion unit to map directional acceleration to pitch, which worked as though there was a keyboard wrapped around each beach ball.
HOW IT WORKS
The Beach Ball Sensor
Each beach ball has a sensor package in a custom made mechanical holder that is held captive by a carabiner. For sensing and communication, we used an Arduino with WiFi, a nine degree of freedom inertial motion sensor that detects acceleration in X, Y, and Z directions, rotation about X, Y, and Z directions, and magnetic field in X, Y, and Z directions. We also opted for an external antenna for added WiFi robustness.
This package communicates raw sensor data via OSC messages over WiFi to a computer that processes the sensor data. A USB battery designed to recharge your smartphone on-the-go powers the hardware and provides enough energy to run each system for about 14 hours—more than enough for an entire day of uninterrupted play.
Signal Processing and Gesture Recognition
On the computer, Max processes the streaming sensor data, recognizes gestures, and outputs a MIDI note with velocity whenever it detects a beach ball impact gesture. Ableton Live takes the MIDI data from all the beach balls, filters the notes to work with the current harmony, and triggers instrument sound.
We wanted to be certain that participants could hear how they were changing the music, so while the background music played evenly throughout the installation, each individual beach ball generated a sound in the speaker closest to it—kind of like stage monitors that bands use playing live. That feedback encouraged everyone to play more.
Collaborative Music Design
To set the stage for musical confidence, we wrote algorithmic music that acted as a backing track, so no one felt the pressure to play the first note. Depending on where a beach ball was hit, one of 15 possible chromatic notes was generated. So, we filtered the beach ball notes so that they always fit consonantly with the backing track—you couldn’t play a “wrong” note. Even still, six different instruments playing over background texture can be too dense. So we opted for a section approach—three sets of two instruments. A trio with backing felt like the right balance of individual voice without cacophony.
When we got to the venue, we had to invent a rigging system to get the beach balls to hang at the right place and height. In a divine moment of MacGyver-like inspiration, Diem Ho devised a hierarchical beach ball wiring system that could be lowered and raised quickly with precise control. When we were swarmed with little kids, this system allowed us to raise the beach balls just enough so people could see little kids behind a beach ball and avoid hitting them; things became a lot safer.
The most important design practice throughout the entire process was to stay nimble and improvise. We repurposed problems into constraints that catalysed our evolving platform. In a sea of unknowns, improvisation allows you to find generative ways to translate even the simplest knowns into a compass toward meaning, and have fun doing it.