SofiArt

Musical Glove Evolution

Click to watch demonstration

I recently developed a software instrument in Max for Live (M4L) that enabled me to perform with the musical glove.  The sounds are based on a custom pythagorean tuning.  At the moment the instrument uses three continuous controllers and two digital switches.  It is challenging to establish a steady stream of sensor data, so I programmed the flakiness of the glove into controllers that didn’t mind some randomness.  The result is quite interesting as long as this doesn’t happen with the wrong parameter.  The next step is to integrate interactive visuals with the musical glove.

The latest version of the Musical Glove  uses continuous controllers and switches affecting parameters such as rhythm, pitch, and delay. The gloves went through multiple design variations.  In the current design I am using soft switches made of conductive fabric and tightly sewed the remaining wire wrap onto the surface of the glove.  Other newer variations of the glove using different conductive fabrics have failed to perform better than the original.  As I found out some conductive fabrics, especially stretch fabrics loose their conductivity with heat causing breaks in the lines.  A new iteration of the glove is currently in development.  The musical gloves were shown at a performance with the band Machine 475 at the Gulu-Gulu Cafe, in Salem, MA, March 26th.

Musical Glove Prototype

Click to watch
Prototype

A Computer Vision Installation

Light Identity is a software mirror installation that plays on the concept of multiple-fragmented identity. It explores individual identity as multiple reflections of subjective diffracted realities of the fragmented self in the perceptible world of the electronic age. It attempts to prompt the question of identity in the mind of the passer-by in a light and playful way, as images of themselves advance ahead of the subject or follow behind. The mirror reflects individual identity as multiple electronic light objects, alienating and dehumanizing individual identity, and, at the same time representing the multi-dimensionality of identity and the self. Different aspects of identity might be physical, mental, psychological, social, spiritual, economic, artistic, ethical.

Why interactive?
The experience is interactive, and interdependent on the presence of the spectator who also becomes the performer in the installation. The otherwise empty physical space of the installation becomes alive only with the involvement of the audience. Under the light of surveillance, the participator realizes that he/she is both the one that looks and the one being looked at. The boarders between art and life, the artist and the audience are blurred. The question of who the artist is becomes less important as interactive art is as much about the artist as it is about the people who actively or passively interact with the art.

The Inspiration

The inspiration for the aesthetic of the effect came from my friend’s Lori Napoleon’s Diffraction lenses and Laser effects – see Lori’s blog http://itp.nyu.edu/blogs/lan274 and web site http://www.subk.net/holoindex.html. I had always wanted a 360 degree projection in physical space with no screen, and Lori was doing just that with lasers. In this project I achieved a 360 degree projection using a servo motor. I now want to transform the installation to a 180 degree projection. The blue green color scheme resembles the Mediterranean sea and also green lasers.

The Process

Shiffman’s mirror effect triggered the idea of imitating light diffraction and the code was perfect for a diffraction effect! I combined the diffraction effect with Golan Levin’s brightness thresholding for background subtraction to achieve the aesthetic I was looking for. Below are the steps involved in completing the project:

1_Designing the raw effect using Shiffman’s mirror example with processing.

2_Visualizing the final effect in photoshop.

3_Background subtraction using brightness thresholding.

4_Combining the effect with background subtraction.

5_Replicating the image into a multiple effect.

Lighting is key!
When I tested it at home against a white wall and using a desk lamp, I knew that the code basically worked. The process of battling with the code to refine the effect, was a mystifying experience, as unlinke any other visual media I had used before, I had no real control over it. Attempting to combine serial communication with video was a challenging mission. Due to partly my ignorance of what this might entail, and partly my confidence with video and lighting, I was determined to continue. The code, succeeded and broke 4 times for no apparent reason except that serial and video are touchy with each other. The things I tweaked to fix the code in the end did not make any sense to me. For example, changing the order of declaring my variables in setup. Amazingly, and as I had heard usually happens, it all came together the night before my presentation.

Physical Computing

1_Using a camera for video tracking x and y data input.

2_Mapping x and y data into arduino using serial communication.

3_Servo motor to control the output movement of the mirror.

The installation

For motion tracking and background subtraction I used a black background, IR light, and a night vision camera. A mirror is mounted on a servo motor projecting an image around the room, according to the horizontal movement of the subject in the camera frame.

Sounds created in the Frameworks for Interactive Sound class, at the Interactive Telecommunications Program, NYU 2008.

• Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Time stretched sound loops generated in MAX/MSP.

• Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Manipulation of original recorded sound of an MTA train ride, until it disintegrates to noise. The sound is taken from my video Transtrain.

• Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Experimenting with intervals on a virtual keyboard in MAX/MSP.

Homemade Sound Objects Uncut

• Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Sound created with home objects and voice, using MAX/MSP and a wireless lavaliere.

Click to watch

Click to watch

Reel produced at MS Viewpoint Ltd., a video production house specializing in HD TV commercials.

Click to watch

Reel produced at Boris FX, Inc., a developer of plug-in special effects and titling applications for non-linear editing and compositing systems.