An interactive installation that explores the effect
of auditory communication on our psyche.
Concept:
Sound is thought of as a transitory medium. As Eric Dolphy said at the end of one of his recordings. "Once it is gone, in the air, you can never capture it again". Yet is has an immense power to move people. Sound's temporal nature communicates with us on a different level than visual media. Some suggest that it is because of its temporal nature that it effects us on such an emotional level. Emotions -like sound- are constantly changing and so it makes sense that sounds and particularly music can be seen as a very effective medium for communicating emotional messages.
Despite what Eric Dolphy says, sound can be captured and re-experienced. Not only can we record it and play it back, we as humans can store the event in our own memory. Our recollection of the sound, gives it a semi-perminance within us. This is what I am interested in. The long-term effect of a temporal experience, on human psyche.
When we have conversations with one another we communicate a huge number of ideas and messages. While we may forget what we say to someone; the conversation, or just a few words of the conversation, can make a huge impact on that person, so much so that they may remember those words for the rest of their life.
Words we speak should be thought about just as much as the words we write. Although there is no visual record of what we say there is a record in the consciousness of those we talk to. I want to make people more aware of the power of spoken words and show that what we say lasts in other people's memories. In a country that is already prone to an oversimplification of language, I hope that my thesis will inspire those who experience it, to converse with greater mind on the meaning and implications of what they say.
The installation:
Communicative Environment will show the impact of aural communication on our psyche, by creating a physical space with an auditory memory. The participant will wear a helmet with a wireless mic and headphones. A custom-coded computer vision system will track the participant in an empty physical space. The system will always know where the mouth of the participant is in three dimensional coordinates. When the participant says something their words will be recorded and the location marked in 3D coordinates. As the participant walks around, they will hear all the other messages left by others, played back spatialized and it will sound like people talking all around the room. To listen to individual messages the participant can locate the sound source in the space and hear it close up. The effect of hearing messages located in 3D will be that it sounds like there are people talking in the room. This combined with the realization that what you say will remain in the space will make the participant feel that what they say will have a lasting effect.
Early Sketches
The technology:
A computer running a c++ computer vision app that I am developing will track infra-red LEDs on top of the helmet, using a camera mounted from the ceiling of the space. The shape of the LEDs will be analyzed and used to determine the position and angle of the head. Another camera with a side-on view will allow the vision system to determine the vertical position of the mouth from the floor of the space. This data will be fed into the audio positioning system that will be used to record and playback samples based on the position. The audio spatialization will be handled by an 3D audio library called OpenAL.
The helmet top down:
