S.J.Zhang
 
 

School PROJECT

Islands of Sound

I collaborated with Ellie Lin to create this online/offline zen experience. We set out to create an interactive interface where users can touch different objects, listen to their matching sounds and see a projection of graphics around them reflecting their properties. We also want to give all the users ability to take the sounds with them, so we designed an App for mixing ambient sounds.

 

Role

Programming

Physical Computing

Play Testing

Fabrication

TOOL

p5.js

Sketch

Conductive Paint

Arduino

Isadora

TIME

2018.11 - Now

Ongoing


 
 
Header.png
 
Screen Shot 2018-12-10 at 10.37.11 PM.png
 
 

Idea + Inspiration

The project started as we imagine a physical interface that user can touch with hand and compose music and graphic. Inspired by sculptures of Isamu Noguchi, works of Arnout Meijer Studio and creations of Olafur Eliasson, we wanted to combine an artful composition of natural materials with programmed lighting effects and a mix of natural and synthetic sounds.

 
 
 
 Credit to Cereal Magazine

Credit to Cereal Magazine

 
 
 Credit to Arnout Meijer Studio

Credit to Arnout Meijer Studio

 
 
 
 Credit to Olafur Eliasson

Credit to Olafur Eliasson

 
 
 
 

Designing the Installation

To create a complete sensory experience. We decided to start with the physical objects. There are mainly tree components to the installation.

  • The touch-sensitive physical interface composed of different objects.

  • The sounds triggered by users touching objects.

  • The graphics accompanying the sounds.

We drew out a system diagram to better dissect what we need to accomplish separately.

 
 
 
System Diagram.png
 
 
 

We then narrowed down to the few objects that we want to use on the interface: wood, rock, bronze, glass, string.

We decided to draw grids of conductive paint over the likes of wood and rock to make the non-conductive objects touch sensitive. We explored two options connecting the signal to the microcontroller.

  • We could draw paint over the board and gather at the end of the board to wires. Lines can also be decorative.

  • We could draw the paint to the back of the objects, and connect to wire there and wire all the connection behind the board.

We went with option ii later due to conductivity constrains with the paint. The charges don’t travel long distance very well.

 
 
Setup Sketch.png
 
 
 

Process: Prototyping & User Testing

 

Capacity Sensing

We decided to use capacity sensing to recognize if someone touches the object. User touching an object changes the capacity value we receive on the analog input pin. Even when the user is not directly touching the conductive area, as long as the touched point is not too far from the conductive material wired to analog input pin, we can get a distinctive value difference.

 
 Testing conductive paint for physical user input.

Testing conductive paint for physical user input.

 
 
IMG_7112.jpeg
 
 

Playtesting

We put one of each our materials on a white foam board and stuck it on the wall. We observed how user interact with the objects and interviewed them of their expectations. They interaction validated a few of our hypothesis, while some ideas inspired us new ways to improve the experience.

 
 

Volume Control

Though initially, the linear element on the board was only there for visual hierarchy so that the composition can be more balanced. Through our playtesting, we found that a lot of people assume the linear element on the board is a slider. Some assume that the slider is for audio volume. Some even suggested that we could project volume levels directly on to the linear element to be visual feedback when interacted.

 
line_element.gif
 
 
 

Fabrication

 

As we explore different materials used for the board. We decided to add a bluetooth chip to our micro-controller so that we only need a power cord into the board. This configuration will also make it easier to move the board to different places. For a complete experience, we only need to plug in the power and map the projection. While a finished board itself with power can also perform a sound-only mode.

 
 
 
 
 
 
 

The Board

Since we need to mount objects on to the board, the board needs to be sturdy. Both acrylic and wood works for the purpose, but since we also need a depth to hide circuits and speakers in the back, sth that comes with a hollowed depth is gonna be helpful. Luckily we found a Lazy Susan at the size we need, with a removable bearing base.

 
 
 
 
 
 

The Cover

We found the perfect fabric to cover the surface, a light grey wool with a slight linen-like texture that is soft and stable. We took inspiration from mattress covers’ construction and sewn a cover with elastic back for the board. When it is mounted or hung against the wall, there is only one noticeable seam around the edge and we have a pretty clean side. There are more seams on the cover, but they are mostly along the grain so barely noticeable.

 
 
 

The Interface

When we are constructing the real interface with real objects at scale, we asked a few musicians to place their ideal compositions. We were told that if we want user to actually remember the sounds associated with the objects and be able to purposefully compose something, we should go for a linear composition reminiscent of piano keyboard and other MIDIs.

We layered a few object to give them some height and now the board looks like a landscape from the side.

 
5.jpg
 
 

Volume Control

Our original idea, was the users can slide their finger along the metal rod to adjust volume. Due to the unpredictable nature of capacity sensing, we weren’t able to get accurate data from the sensor on where the contact points is between the rod and the fingers. We opted for a different solution where the rod is separated into two parts, touching one of them would increase the volume and the other to decrease. We bended metal strip to fix the rod onto the board.

 
IMG_7383.jpeg
 
 

* Fabrication in process. More updates available soon.