SofiArt

Rainbow Resonance is a computer vision installation that generates colors and musical compositions of the equivalent sound frequencies of the spectrum, according to the motion of the participator.

Rainbow Resonance at Ms Viewpoint Ltd, Nicosia, July 8th 2010.
Click to watch video

The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements.  Anyone can potentially create a personal experience within the installation. The color scheme is based on the visible light spectrum and is also inspired by the chakras (energy points on the body).

Rainbow Resonance at ARTos Cultural and Research Foundation, November 2010.
Click to watch video

Color to Sound Mapping

Rainbow Resonance the New York Hall of Science, October 2008, Spring 2009.

Light wavelengths are transposed 40 octaves below the visible light spectrum, into audible sound frequencies and mapped to an 8 tone musical scale based on the Pythagorean tuning system.    My intention was to use natural rhythms as a reference for an alternative intonation system to represent colors.   By trying to map colors to sounds I was surprised to discover that a scale analogous to the equal tempered scale was possible to design, matching the light wavelengths in the spectrum, to a single octave in the sound frequency spectrum.The resulting eight tone scale corresponds to the “white keys” on the piano, and represents the seven predominant colors of the light spectrum.

In the color to sound mapping of Rainbow Resonance, a universal correspondence of sound to color is sought in order to produce an ultimate synaesthetic experience that transcends the participant into a meditative state.  This interactive performance seeks an alignment of the chakras through sound and light vibrations, while the Chi Kung meditation exercise “Carrying the Moon” activates the chakra energy points to enable the balancing of the energies of the body.  Through dance, play and movement, the audience may be entertained, and healed from the stress of the day.

User Experience

Visitors experiencing Rainbow Resonance at Ms Viewpoint Ltd, Nicosia, July 8th 2010.

A lingering sound invites the visitor to enter the space of the installation. Upon entering, the visitor faces a mirror image of him/herself projecting the colors of the rainbow, which are mapped to the figure according to energy points on the body (chakras).  Simple movements generate harmonic intervals, while the equivalent colors brighten up.  Movement along the vertical axis iterates through the color musical scale, while movement along the horizontal axis adds richness to the sounds with additional chords.  An all encompassing composition of the frequencies occurs when the arms are extended upwards towards the sky. The movement is reminiscent of an exercise in Chi Kung (Qigong), an ancient Chinese method of healing and strengthening the body, mind and spirit, called “Carrying the Moon”. “Carrying the Moon” is an exercise that focuses on enhancing youthfulness by keeping the spine young.

Children playing at Rainbow Resonance ARTos Foundation, Nicosia, November 3rd 2010.

More info on Rainbow Resonance at ARTos Foundation, Nicosia

The target audience is everyone who enjoys playing with colors and sounds.  This interactive experience can be modified to benefit children with disabilities, and can be used as an educational and recreational tool for movement through play and dance.

Special children enjoying Rainbow Resonance at Ms Viewpoint Ltd, Nicosia, July 13th 2010.
Click to watch video

more info on Rainbow Resonance at MS Viewpoint, Nicosia

Chakra associations

In Rainbow Resonance the chakra energy points are associated with specific colors as well as sounds and are mapped to different organs, vertically along the y-axis of an upright body, beginning with red at the lower abdomen area, orange at the navel area, yellow around the stomach, lime-green at the heart , blue/turquoise at the throat, and purple at the forehead and above.

Chi kung – A healing exercise

Chi Kung began when prehistoric man discovered how to manipulate breath, as a form of cosmic energy, and according to Chinese records by 2700 BCE it had become an important aspect of chinese medicine. The earlier type of Chi Kung was probably a form of meditative dance which encouraged energy balance in the body. Chinese martial arts, especially Shaolin Kungfu make use of Chi Kung to enhance fighting abilities. Carrying the Moon is one of the best exercises in Chi Kung and can be practiced in many styles. A Chinese saying tells us “You need not worry about growing old, so long as your spine is young”.  Carrying out the exercise daily enhances youthfulness and health at 60 and beyond and strengthens the kidneys, thus enhancing sexual vitality.

A Musical Interface – The Body as an Instrument

Arianna Economou dancing at Rainbow Resonance, Ms Viewpoint Ltd, Nicosia, July 8th 2010.

Rainbow resonance focuses on a human-centered experience that uses the body as the ultimate instrument of expression through dance, play and movement.  It utilizes ambient intelligence technology (computer vision) to empower the participant by enabling them to playfully interact with a software mirror that generates sounds and visuals.

Alexis Vassiliou dancing at Rainbow Resonance, ARTos Foundation, Nicosia, Nov 3rd 2010.
Click to watch video

Technology

A night-vision video camera acts as a sensor to detect motion in space and background subtraction is used to isolate human presence from the environment.  Motion tracking is used to map the “zones” of movement in the space and on the screen.  The “zone” with the most vivid motion determines the predominant sound generated by the participant.  Other hardware components included in the space are infrared lights, a laptop, speakers, a projector and a human-size screen.  MAX/MSP/Jitter and OpenGL are used for motion tracking and visual effects, and, Max for Live (M4L) executes musical sounds using samples, AM and FM synthesis.

Esoteric Music Theory

The esoteric aspect of music theory runs deep and wide. In this project I have only attempted to scratch on the surface of this complex field of study. It is admittedly intriguing and inspiring to explore correlations between the physical world and symbolisms that pertain to the metaphysical realm, both part of nature.

Inspiration

“Through Vibration comes Motion
Through Motion comes Color
Through Color comes Tone”
Pythagoras

Pythagoreans believed that the universe is constructed according to the laws of the divine science of mathematics and that mathematics is the bridge between the visible and invisible worlds.  In the Pythagorean tradition color is bound by the laws of harmony as all else in nature.  Pythagoras was convinced that by aligning the forces of light and sound human beings could be healed from ailments, and he carried out the practice of “Medicinal Music”.

Background

The background for this project starts from an interest in mapping colors to sounds and vice versa, to examine the different emotional, mental or spiritual states they might evoke. I had always wanted to “hear the colors” and “see the sounds”, and wondered whether colors or sound frequencies actually depict or evoke specific emotions or thoughts. The idea of using the chakras as a color reference was inspired by a preoccupation to “see the invisible” by photographing people’s auras. The chakras are associated with specific colors as well as sounds. However, different theories that associate sounds to the chakras and their equivalent colors have not reached a consensus. Today, scientists use sensitive devices such as SQUID (Superconducting Quantum Interference Device) instrument technology to map the ways diseases alter biomagnetic fields around the body. Tests are being carried out to measure the magnetic fields around energy points on certain Qiqong masters.

Rainbow Resonance at ARTos Cultural and Research Foundation, Nicosia

Rainbow Resonance at MS Viewpoint, Nicosia

Rainbow Resonance at Harvestworks, New York

Rainbow Resonance at the ITP Spring Show 2008

Click to watch Rainbow Resonance at the ITP Spring Show 2008.

Preliminary worksheets

Tags: computer vision, interactive performance, Rainbow Resonance

This entry was posted on Saturday, November 7th, 2009 at 11:40 am and is filed under interactive, Rainbow Resonance. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.