Echolations
The Op on Screen Interactive Festival
Arleen Schloss and Emma Zakarevicius: Echolations
Saturday, October 24 at 2PM – New York Public Library – Hamilton Fish Park
Echolations, a performance-sound environment for electronically processed voices. Arleen Schloss & Emma Zakarevicius will present a collaborative performance piece for voices and electronic echolations investigating the primitive desire with alphabetic sanskrit vocal sounds and digital techno-poetics (signs) as a mode of communication and connection.
Arleen Schloss & Emma Zakarevicius performing Echolations
Arleen Schloss is a performance art pioneer, video artist, sound poet, curator, and founder of the A’s Salon in the late 1970, which became one of the most influential places in the Downtown art and music scenes. Schloss has exhibited and performed extensively throughout Europe, Asia and the U.S. including the Museum of Modern Art in New York. Emma Zakarevicius is a multi-disciplinary artist whose primary interest is how meaning is transmittable through various artistic mediums in order to incite communication, connection and participation. She is a performer, singer/songwriter and visual artist currently enrolled for a Masters in Media Studies at The New School.
Op On Screen 2009
The Op on Screen Interactive Festival
Four events of two generations of women artists/composers and collaborators
New York Public Library – Hamilton Fish Park
415 E Houston St.@ Avenue D
FREE
Sofia Paraskeva: Rainbow Resonance – Thursday, October 15 at 6PM
Rainbow Resonance is a computer vision installation that generates colors and musical sounds of the equivalent sound frequencies in the electromagnetic spectrum, according to the motion of the participator. The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements.
Sofia Paraskeva: Rainbow Resonance
Lesley Flanigan: Voices for Speakers – Saturday, October 17 at 2PM
Voices for Speakers uses vocalizations and feedback technology to create an interactive sound environment. This performance of Lesley’s work will include some new sounds using the dynamic voices of sopranos Sarah Moulton and Pamela Stein along with the video artistry of Luke DuBois.
Lesley Flanigan, Sarah Moulton and Pamela Stein in performance
Sofia Paraskeva and Elodie Lauten: The Trickster – Thursday, October 22 at 6PM
The Trickster is a collaborative sound and interactive installation with choreography by Khoi Bao Le and visuals by Sofia Paraskeva. This performance is an offshoot from Lauten’s new work The Two-Cents Opera, involving interactivity between movement, music, and image projections triggered by dance and wearable components.
Khoi Bao Le performing in The Trickster
Arleen Schloss and Emma Zakarevicius: Echolations – Saturday, October 24 at 2PM
Echolations, a performance-sound environment for electronically processed voices. Arleen Schloss & Emma Zakarevicius will present a collaborative performance piece for voices and electronic echolations investigating the primitive desire with alphabetic sanskrit vocal sounds and digital techno-poetics (signs) as a mode of communication and connection.
Arleen Schloss & Emma Zakarevicius performing Echolations
About the Artists
Sofia Paraskeva, Lesley Flanigan and Emma Zakarevicius are active emerging artists whose work has been presented in the US and internationally. Also featured is a young emerging choreographer from Viet Nam, Khoi Bao Le. Arleen Schloss is a performance art pioneer, video artist and founder of the A’s Salon in the late 1970, which became one of the most influential places in the Downtown art and music scenes. Elodie Lauten is a leading multimedia artist and music technology educator with credits at the New York City Opera, Lincoln Center and the Whitney Museum, recently featured at the Theater for the New City and the Women Forward exhibition praised by Mayor Bloomberg in Williamsburg.
The Op on Screen Festival is presented by Lower East Side Performing Arts, with support from Lower Manhattan Cultural Council Fund for Creative Communities for the third consecutive year. It is free and open to the public.
Rainbow Resonance – Computer Vision Performance
Rainbow Resonance is a computer vision installation that generates colors and musical compositions of the equivalent sound frequencies of the spectrum, according to the motion of the participator.
Rainbow Resonance at Ms Viewpoint Ltd, Nicosia, July 8th 2010.
Click to watch video
The audience is encouraged to participate in a playful performance that engages the body to produce colorful images and sound resonances through simple movements. Anyone can potentially create a personal experience within the installation. The color scheme is based on the visible light spectrum and is also inspired by the chakras (energy points on the body).
Rainbow Resonance at ARTos Cultural and Research Foundation, November 2010.
Click to watch video
Color to Sound Mapping
Rainbow Resonance the New York Hall of Science, October 2008, Spring 2009.
Light wavelengths are transposed 40 octaves below the visible light spectrum, into audible sound frequencies and mapped to an 8 tone musical scale based on the Pythagorean tuning system. My intention was to use natural rhythms as a reference for an alternative intonation system to represent colors. By trying to map colors to sounds I was surprised to discover that a scale analogous to the equal tempered scale was possible to design, matching the light wavelengths in the spectrum, to a single octave in the sound frequency spectrum.The resulting eight tone scale corresponds to the “white keys” on the piano, and represents the seven predominant colors of the light spectrum.
In the color to sound mapping of Rainbow Resonance, a universal correspondence of sound to color is sought in order to produce an ultimate synaesthetic experience that transcends the participant into a meditative state. This interactive performance seeks an alignment of the chakras through sound and light vibrations, while the Chi Kung meditation exercise “Carrying the Moon” activates the chakra energy points to enable the balancing of the energies of the body. Through dance, play and movement, the audience may be entertained, and healed from the stress of the day.
User Experience
Visitors experiencing Rainbow Resonance at Ms Viewpoint Ltd, Nicosia, July 8th 2010.
A lingering sound invites the visitor to enter the space of the installation. Upon entering, the visitor faces a mirror image of him/herself projecting the colors of the rainbow, which are mapped to the figure according to energy points on the body (chakras). Simple movements generate harmonic intervals, while the equivalent colors brighten up. Movement along the vertical axis iterates through the color musical scale, while movement along the horizontal axis adds richness to the sounds with additional chords. An all encompassing composition of the frequencies occurs when the arms are extended upwards towards the sky. The movement is reminiscent of an exercise in Chi Kung (Qigong), an ancient Chinese method of healing and strengthening the body, mind and spirit, called “Carrying the Moon”. “Carrying the Moon” is an exercise that focuses on enhancing youthfulness by keeping the spine young.
Children playing at Rainbow Resonance ARTos Foundation, Nicosia, November 3rd 2010.
More info on Rainbow Resonance at ARTos Foundation, Nicosia
The target audience is everyone who enjoys playing with colors and sounds. This interactive experience can be modified to benefit children with disabilities, and can be used as an educational and recreational tool for movement through play and dance.
Special children enjoying Rainbow Resonance at Ms Viewpoint Ltd, Nicosia, July 13th 2010.
Click to watch video
more info on Rainbow Resonance at MS Viewpoint, Nicosia
Chakra associations
In Rainbow Resonance the chakra energy points are associated with specific colors as well as sounds and are mapped to different organs, vertically along the y-axis of an upright body, beginning with red at the lower abdomen area, orange at the navel area, yellow around the stomach, lime-green at the heart , blue/turquoise at the throat, and purple at the forehead and above.
Chi kung – A healing exercise
Chi Kung began when prehistoric man discovered how to manipulate breath, as a form of cosmic energy, and according to Chinese records by 2700 BCE it had become an important aspect of chinese medicine. The earlier type of Chi Kung was probably a form of meditative dance which encouraged energy balance in the body. Chinese martial arts, especially Shaolin Kungfu make use of Chi Kung to enhance fighting abilities. Carrying the Moon is one of the best exercises in Chi Kung and can be practiced in many styles. A Chinese saying tells us “You need not worry about growing old, so long as your spine is young”. Carrying out the exercise daily enhances youthfulness and health at 60 and beyond and strengthens the kidneys, thus enhancing sexual vitality.
A Musical Interface – The Body as an Instrument
Arianna Economou dancing at Rainbow Resonance, Ms Viewpoint Ltd, Nicosia, July 8th 2010.
Rainbow resonance focuses on a human-centered experience that uses the body as the ultimate instrument of expression through dance, play and movement. It utilizes ambient intelligence technology (computer vision) to empower the participant by enabling them to playfully interact with a software mirror that generates sounds and visuals.
Alexis Vassiliou dancing at Rainbow Resonance, ARTos Foundation, Nicosia, Nov 3rd 2010.
Click to watch video
Technology
A night-vision video camera acts as a sensor to detect motion in space and background subtraction is used to isolate human presence from the environment. Motion tracking is used to map the “zones” of movement in the space and on the screen. The “zone” with the most vivid motion determines the predominant sound generated by the participant. Other hardware components included in the space are infrared lights, a laptop, speakers, a projector and a human-size screen. MAX/MSP/Jitter and OpenGL are used for motion tracking and visual effects, and, Max for Live (M4L) executes musical sounds using samples, AM and FM synthesis.
Esoteric Music Theory
The esoteric aspect of music theory runs deep and wide. In this project I have only attempted to scratch on the surface of this complex field of study. It is admittedly intriguing and inspiring to explore correlations between the physical world and symbolisms that pertain to the metaphysical realm, both part of nature.
Inspiration
“Through Vibration comes Motion
Through Motion comes Color
Through Color comes Tone”
Pythagoras
Pythagoreans believed that the universe is constructed according to the laws of the divine science of mathematics and that mathematics is the bridge between the visible and invisible worlds. In the Pythagorean tradition color is bound by the laws of harmony as all else in nature. Pythagoras was convinced that by aligning the forces of light and sound human beings could be healed from ailments, and he carried out the practice of “Medicinal Music”.
Background
The background for this project starts from an interest in mapping colors to sounds and vice versa, to examine the different emotional, mental or spiritual states they might evoke. I had always wanted to “hear the colors” and “see the sounds”, and wondered whether colors or sound frequencies actually depict or evoke specific emotions or thoughts. The idea of using the chakras as a color reference was inspired by a preoccupation to “see the invisible” by photographing people’s auras. The chakras are associated with specific colors as well as sounds. However, different theories that associate sounds to the chakras and their equivalent colors have not reached a consensus. Today, scientists use sensitive devices such as SQUID (Superconducting Quantum Interference Device) instrument technology to map the ways diseases alter biomagnetic fields around the body. Tests are being carried out to measure the magnetic fields around energy points on certain Qiqong masters.
Rainbow Resonance at ARTos Cultural and Research Foundation, Nicosia
Rainbow Resonance at MS Viewpoint, Nicosia
Rainbow Resonance at Harvestworks, New York
Rainbow Resonance at the ITP Spring Show 2008
Click to watch Rainbow Resonance at the ITP Spring Show 2008.
Preliminary worksheets
Musical Gloves
Musical Glove Evolution
I recently developed a software instrument in Max for Live (M4L) that enabled me to perform with the musical glove. The sounds are based on a custom pythagorean tuning. At the moment the instrument uses three continuous controllers and two digital switches. It is challenging to establish a steady stream of sensor data, so I programmed the flakiness of the glove into controllers that didn’t mind some randomness. The result is quite interesting as long as this doesn’t happen with the wrong parameter. The next step is to integrate interactive visuals with the musical glove.
The latest version of the Musical Glove uses continuous controllers and switches affecting parameters such as rhythm, pitch, and delay. The gloves went through multiple design variations. In the current design I am using soft switches made of conductive fabric and tightly sewed the remaining wire wrap onto the surface of the glove. Other newer variations of the glove using different conductive fabrics have failed to perform better than the original. As I found out some conductive fabrics, especially stretch fabrics loose their conductivity with heat causing breaks in the lines. A new iteration of the glove is currently in development. The musical gloves were shown at a performance with the band Machine 475 at the Gulu-Gulu Cafe, in Salem, MA, March 26th.
Musical Glove Prototype
Click to watch
Prototype
Light Identity
A Computer Vision Installation
Light Identity is a software mirror installation that plays on the concept of multiple-fragmented identity. It explores individual identity as multiple reflections of subjective diffracted realities of the fragmented self in the perceptible world of the electronic age. It attempts to prompt the question of identity in the mind of the passer-by in a light and playful way, as images of themselves advance ahead of the subject or follow behind. The mirror reflects individual identity as multiple electronic light objects, alienating and dehumanizing individual identity, and, at the same time representing the multi-dimensionality of identity and the self. Different aspects of identity might be physical, mental, psychological, social, spiritual, economic, artistic, ethical.
Why interactive?
The experience is interactive, and interdependent on the presence of the spectator who also becomes the performer in the installation. The otherwise empty physical space of the installation becomes alive only with the involvement of the audience. Under the light of surveillance, the participator realizes that he/she is both the one that looks and the one being looked at. The boarders between art and life, the artist and the audience are blurred. The question of who the artist is becomes less important as interactive art is as much about the artist as it is about the people who actively or passively interact with the art.
The Inspiration
The inspiration for the aesthetic of the effect came from my friend’s Lori Napoleon’s Diffraction lenses and Laser effects – see Lori’s blog http://itp.nyu.edu/blogs/lan274 and web site http://www.subk.net/holoindex.html. I had always wanted a 360 degree projection in physical space with no screen, and Lori was doing just that with lasers. In this project I achieved a 360 degree projection using a servo motor. I now want to transform the installation to a 180 degree projection. The blue green color scheme resembles the Mediterranean sea and also green lasers.
The Process
Shiffman’s mirror effect triggered the idea of imitating light diffraction and the code was perfect for a diffraction effect! I combined the diffraction effect with Golan Levin’s brightness thresholding for background subtraction to achieve the aesthetic I was looking for. Below are the steps involved in completing the project:
1_Designing the raw effect using Shiffman’s mirror example with processing.
2_Visualizing the final effect in photoshop.
3_Background subtraction using brightness thresholding.
4_Combining the effect with background subtraction.
5_Replicating the image into a multiple effect.
Lighting is key!
When I tested it at home against a white wall and using a desk lamp, I knew that the code basically worked. The process of battling with the code to refine the effect, was a mystifying experience, as unlinke any other visual media I had used before, I had no real control over it. Attempting to combine serial communication with video was a challenging mission. Due to partly my ignorance of what this might entail, and partly my confidence with video and lighting, I was determined to continue. The code, succeeded and broke 4 times for no apparent reason except that serial and video are touchy with each other. The things I tweaked to fix the code in the end did not make any sense to me. For example, changing the order of declaring my variables in setup. Amazingly, and as I had heard usually happens, it all came together the night before my presentation.
Physical Computing
1_Using a camera for video tracking x and y data input.
2_Mapping x and y data into arduino using serial communication.
3_Servo motor to control the output movement of the mirror.
The installation
For motion tracking and background subtraction I used a black background, IR light, and a night vision camera. A mirror is mounted on a servo motor projecting an image around the room, according to the horizontal movement of the subject in the camera frame.
Computer Vision Performance – Essay
The possibilities of image processing technologies, particularly data visualization such as computer vision applications or software “mirrors”, when combined with physical computing, are intriguing endless manifestations of creative ideas. Computer vision software is a fascinating field of experimentation for artists. Using the latest technological tools and micro chips such as microcontrollers, interactive screens and installations become alive; art that involves interaction with subjects who happen to walk, wander or perform in front of the camera “mirror”, occurring over a continuum of real time. In the ultimate interactive environment the participator, the person who is being mirrored, can affect the images and/or sounds experienced with real time feedback. Essentially the participator brings the installation to life by “performing” in front of the camera producing individualized unique experiences. Wikipedia, defines computer vision systems as “the scientific study of the theories and techniques for building systems that perform “perception” using images or multi-dimensional data”.
The link between performance art and interactive screens or computer vision is apparent. A reference to the history of minimalism and performance art as it was shaped through the 1960s is necessary to build a connection between past, current and future trends. In the 1960s the performance ideas of John Cage, Merce Cunningham, as well as the “happenings” of Alan Kaprow, and the event-based works of Claes Oldenberg and the Fluxus artists, greatly influenced the artistic scene of New York. A “minimalist aesthetic” was felt throughout the arts including cinema, new theater, conceptual art, performance, and video art. The term Happening, was coined by Alan Kaprow in the late 1950s to describe event-based performances shaped by the participation of the audience that occurred in physical spaces such as abandoned factories, lofts, parks, buses and so no. John Cage’s experimental performances at Black Mountain College in the late 1940s constitute the first Happening events. The minimalist aesthetic rejected any sense of authorship of the artwork and emphasized art as a collaborative, democratic social experience, through an unmediated art, in which the participation of the viewer was as important as that of the artist in the completion of the art work. Influenced by Cage’s indeterminacy, chance operations and audience involvement, Kaprow among others, attempted to eliminate any distinction between audience and performer all together. Kaprow believed that “art is a continual work-in-progress, with an unfolding narrative that is realized through the active participation of the audience.” Furthermore, the theories of Marshal McLuhan whose philosophy may be summed up in the phrase “the medium is the message”, and Buckminster Fuller, prompted artists to seek for spiritual transcendence, by examining the “capabilities” and physiology of the medium. Video artists believed that their formal effects through video art would neutralize mass media by exposing the manipulative intent of television, and thus subvert cultural attitudes towards the mainstream altogether.
The great video art pioneer and fluxus artist Nam June Paik accurately predicted that “someday artists will work with capacitors, resistors and semi-conductiors as they work today with brushes violins and junk.” Improvisational performance is indeed the most “unmediated” form of art, happening in the real now, and representing nothing but itself. Data visualization tools are closer than ever to becoming full interactive experiences, where art becomes real life. With computer vision tools in our hands, it is a challenge to create artworks that become subjective when a participator takes an active role in the piece. The experience is not unmediated because the result is always controlled within certain parameters and variables. But there is still an element of unpredictability and improvisation in the unique outcome of every single subjective experience, when a participator engages in the art work. I am not particularly concerned with providing the illusion to “a user” or participator that they are creating their own effects, or sounds through an art piece I’ve made. The fascinating aspect of computer vision and interactive screens is in my opinion their existence in real time, the very fact that they constitute performances that happen in real life. Whether the viewer, or participator just cruises through the piece or takes a more active part in it, is not that important. What matters is that an interactive experience is essentially a performance happening in the real now, and is real life in itself. Thus art and life are merged.
During a lecture on one of the drivebys at ITP, Marius Watz, an artist and curator who has been working with generative art since the mid-1990’s, showed us his work which uses abstract software compositions in 2D and 3D space. According to Watz generative systems encode subjective aesthetic principles into an external system, such as a piece of software or hardware, or even a completely analog system. In Watz’s opinion this approach provides a computational model of creativity, making it popular with a new generation of digital artists and designers who are turning to code to create new forms of expression. I think that Watz has a point in claiming that artists turn to code, for creative expression, but perhaps the motive behind that is not an examination of the form of code and what it can do, but an abidance with the trends. It’s cool to be working with code in 2007. New technologies are always fascinating to experiment with, especially technologies that are open-source, or free as is processing. Furthermore, artists who explore code are not necessarily concerned with content. In the 1990s digital artists were examining the form of the medium and its capabilities, as artists did before with video. I believe content can be enriched if code is combined with other media, or ideas beyond the scope of exploring code for its own sake. For example, I believe that Watz’s approach to generative art is in sharp contrast to computer vision artists such as Daniel Rozin. Although I enjoyed Watz’s art, I found it somewhat alienating. However, the interactivity of projects such as Rozin’s have an immediacy, a compelling involvement that the passive experience of just simply watching generative art on a screen lacks. Whenever I pass by the Wooden mirror on the 4th floor at ITP, I become aware of my presence in that specific surrounding. The interactive sound and my wooden reflection makes me imagine that the artwork is alive, and there is admittedly a special magic to this “live” real time experience. The wooden mirror has become part of my life.
The struggle between art and technology has never really been resolved until now. Luckily for us, formalist ideologies developed a long time ago by Bauhaus, and Laszlo Moholy-Nagy among others, included scientific experimentation in the art vocabulary. However, artists, critics and people, still debate what is art, and what is not, depending on what medium is used. For example, recently I saw two performances in the east village, which involved visuals and sound. In both of them sound was improvised on the spot. The first piece was generative art and the second one was film. I especially enjoyed the first piece where the abstract digital images imitated real life in a few glimpses, alluding to shots of nature. However, at the end of the performance a rather hostile conversation evolved when a member from the audience who is clearly purist in attitude as a painter, reacted about the fact that digital, generative art was used to imitate nature. Interactive art projects are now bridging the gap between art and technology once and for all. This perfect marriage between art and technology is in my opinion the way of the future for many artists.
Finally, I want to relate interactive performance art to the way it capture the attention and brainpower of the participator. In a lecture I attended recently at ITP, Uri Hasson, a postdoctoral fellow at the NYC/CNS (Center for Neural Science) at NYU, presented the results of a study on measuring the effect of films on viewers’ minds, using data visualization (MRI-magnetic resonance imaging). His results illustrated the relative involvement of the brain comparatively in different genres. At one end was no involvement and at the other end was propaganda. Literary films and well told stories as in hollywood seem to surpass other types of genres, with Hitchcock films reaching a brain involvement of 70%. Art films fell rather low on the scale of brain involvement. I would be interested to find out what the involvement of interactive performance art is. It is my suspicion that it would be much higher than art films in general. Perhaps good narrative stories will always be the ultimate form of entertainment for people, whether told through a performance or a screen.
Art was always thought of as a consciousness changing medium. Generative art, or computer art, is no different. Art questions life, whether it’s representational or not, dynamically changing culture. The challenge is to bring art out of the margins and into the mainstream. Computer vision and interactive screens and installations pose the ultimate challenge for artistic expression currently, to me. Interactive experiences are pervasive and immersive when combined with narrative stories. For a while we went into the screens to find art, now we are coming out of the screens into the physical world, via the computer. Interactive art merges art and life, making one aware of the presence of the self in the here and now.
Sound samples
Sounds created in the Frameworks for Interactive Sound class, at the Interactive Telecommunications Program, NYU 2008.
-
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Time stretched sound loops generated in MAX/MSP.
-
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Manipulation of original recorded sound of an MTA train ride, until it disintegrates to noise. The sound is taken from my video Transtrain.
-
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Experimenting with intervals on a virtual keyboard in MAX/MSP.
Homemade Sound Objects Uncut
-
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Sound created with home objects and voice, using MAX/MSP and a wireless lavaliere.
Altered Visions
Experimental 16mm film using time-based manipulation on the optical printer. 6 min, 1999
Anima, Animus
Photis Nicolaou performing with Evangelia Zachariades in Anima, Animus, Dec, 2003
Anima Animus
Experimental dance video – 3.30 min.
Writer, director, art director, editing Sofia Paraskeva
A woman discovers her animus, the male aspects of her inner self that are hiding in the unconscious mind, according to Carl Jung. Her animus joins her in an improvisational dance where they unite against the background of a spiral as they reach “consciousness”.
Thanks to MS Viewpoint Ltd for offering their support, studio and resources to shoot this video.
Bodyparts
Click to watch Experimental video, 1.38 min. Edited on Final Cut Pro, 2007.