The Pulsefield has a growing number of applications (Apps) which define the video, sound, and light experience. With its modular architecture, new Apps can be easily added through Ableton Live, MAX/MSP, Java, or Processing. Here are some of the current Apps, each with a description and short video of a session with several users.
This App generates a Navier-Stokes simulation of a fluid. Imagine the Pulsefield as a pool of water with each person that enters adding a colored oil to the pool along their track. Their movements also disturb the surface (and the prior oil streams) creating currents, dispersion, vortices, etc. This achieved through implementation of a numerical solution (based on work by Jos Stam ) of the Navier-Stokes set of equations with parameters such as diffusion and viscosity controlled by the iPad controller.
The music that accompanies this App is a set of ambient sound synthesizers similar to the Pads App above with parameters of the sound controlled by the movements of the users.
I first saw Osmos as an iOS game ( http://www.osmos-game.com ) built by Hemisphere Games. Using that as inspiration, I built a version where the Pulsefield participants drag planets around the ground stealing mass from each other while exerting gravity. Baby planets are spewed out by moving, which also loses mass.
As a synthetic biologist, I always wanted to incorporate more science into my art. DNA allows users to drag chains of DNA around behind them. If they pull too hard, the chain can be cut (endonucleases), if the 3′ end of their chain hits a 5′ end of a free chain, it may be joined (ligatiion). And if you get a complementary sequence to line up with someone else’s chain, it can hybridize to form a helix!
This App is a game based on the light cycles of the Tron movie and the corresponding arcade games from the 80’s. Each player leaves behind them a trail of colored blocks as they move. If a player runs into their own or another player’s blocks, they “die” — all of their blocks are removed and they start over again. The music that accompanies this mode is generated by the pathways the leave behind which is “played” as follows:
- Each player is assigned a starting pitch.
- On each beat a cursor moves one block ahead on the path.
- If the the path continues straight the same note is played on the next beat; if there is a left turn, the pitch is dropped by one note in the scale, for a right turn the pitch is increased by one note in the scale.
- When the end of a path is reached, the cursor reverses direction; since the turn directions are reversed, the pitch returns to the same starting pitch once the entire path is covered.
As it stands, the audio becomes somewhat cacophonic with several players — reminiscent of early arcade games.
The Guitar App is a simulation of an acoustic Guitar. The Pulsefield has virtual strings and frets of a guitar overlaid on the space. When participants move such that the cross one of the strings, that string is played with a note corresponding to the particular string with the nearest fret to the left of where the crossed being held down. The speed at which they cross the string governs the loudness of the note. The App includes a video representation of the guitar showing the positions of all the people in the Pulsefield with vibrating strings highlighting the notes. The video is further enhanced with a smoke effect, each person (whose color remains the same during their time in the Pulsefield) generates a cloud of rising smoke. Their movements create virtual air drafts that perturb their own and others’ smoke. The sound output is generated from Ableton Live using a guitar synthesizer.
The Pads app is an ambient music generator. Each person that enters the Pulsefield is assigned a particular synthesizer which generates sound continuously as long as they are moving. These synthesizer are each setup to have two input parameters that affect either the pitch or character or the sound — those parameters are controlled by the assigned user’s X and Y position in the Pulsefield. In addition, for some of the sounds, their velocity in the X and Y direction control two additional synthesizer parameters. The visuals for this mode are the same rising smoke generators used in the guitar App.
The Poly App was inspired by James Milton‘s iOS app of the same name. The Pulsefield is mapped to a set of concentric circles that represent 1/16 note divisions. Triggers start from the and radiate outwards and when they intersect a participant’s position a note is fired. Thus, people that are closer to the center have notes generated more frequently. Differences between positions of different people result in changing relation of the timing between their notes being played. Also, the angle of the person from the center (their “clock time”) controls the pitch of the notes they play. Both the video and music follow this model. The sound synthesis still needs more work — Jame’s version still sounds much better!
One final element of Poly which has proved quite popular is a group behavior reinforcement: if 2 or more people hug or stay close enough that there is no visible gap between them, then they are treated as group with “fireworks” at their location and repeated notes being played. Once people discover this they start hugging each other, and forming larger and larger groups.
A Voronoi tesselation or Delaunay triangulation is a division of space based on a set of seed locations such that the points in each region are closer to the seed in that region than any other seed. For the Pulsefield, each person acts as a seed and lines are drawn in the generated video that gives each person their own region. The lines between them are then played as a harp. The system cycles through the dividing lines playing each one a harp synthesizer with a pitch that corresponds to the length of the line.
This mode is visually interesting but the sound gets very quickly complicated making it hard for users to understand the effects of their movements or steer the output toward particular outputs making it difficult to “play” as an instrument.
Dance Dance Revolution
Dance Dance Revolution or DDR is a popular music video game where players stand on a platform and need to step on pads arranged in a cross formation in synchrony with the music and onscreen arrows. The Pulsefield offers a method for many people to play DDR without the need of control pads. Instead the hip motion of the players is tracked while the music and directional arrows are presented on the video screen. Players can be anywhere within the Pulsefield while the App tracks their mean position and their movements. The music/direction material for this App was adapted from materials for the open source re-implemention of DDR, StepMania.
There are a few limitations of this App which need improvement. For example, some of the directions from existing StepMania/DDR programs include multi-steps — such as a split (one foot on each of the two side pads) or one foot on the front pad and one on the back. Since your hip can only move one way at a time, these are impossible to do on the Pulsefield implementation. Also, scoring has yet to be implemented. Despite these limitations and a limited repetoire of songs users of this App enjoyed it immensely.
This was the first App for the Pulsefield. The space within the Pulsefield is virtually divided into an 8 by 9 grid (actually 60 grid squares since some in the corners would fall outside). Each grid position is then mapped to a single scene in Ableton Live. As each person enters they are assigned an individual ID, color, and track, such as guitar, drums, piano, etc. As they move through the Pulsefield the particular scene corresponding to their position is triggered in Ableton. All of the scenes for a song are beat-aligned and chosen such that the music always “works”, yet is clearly under the control of the participants.
There are currently 10 different songs for this mode, each with 6 to 8 tracks, that were composed by Marcos Saenz expressly for the Pulsefield. They range from a set of Gamelan bells recorded in Bali to experimental electronic music. When people enter the Pulsefield, one of the songs is chosen at random and is then fixed for the duration of the interaction. When all participants exit the Pulsefield is a new song selected. It is also possible to choose the song and the individual track assignments through the iPad application.
For this mode, the video can be a simple display of the position of each participant as viewed from above with the virtual grid overlaid showing the triggering. Or it can be coupled with an independent visual display, such as the Navier-Stokes simulation described below.