Pedestal

Pedestal is a small projection on the wall over a bookshelf. The bookshelf is high enough that you can’t really see the bottom of the projection. The closer you stand to the bookshelf, the less you see of the projection.

This projection tells a little story. Several people are milling around. There’s a small pedestal in the center. One person stands on it and everyone else is looking up at him or her. The person on the pedestal is more visible to the viewer, and can ‘see’ the viewer as well.

This piece comes with a little step, like in a library, so you can visit with your people.

It’s You is an interactive storefront-window projection that explores the mechanisms of public behaviors and the line between the real and constructed social actions.

The installation is a rear projection on large storefront window. Human figures crowd around something that they obscure from the pedestrian’s view. When the pedestrian stands behind them, as if to look over their shoulders, they step aside to allow him a view onto what they’re looking at. The pedestrian can now see part of the unfolding scene, and he obscures the view for the other pedestrians; he’s become part of the crowd.

When the pedestrian enters the interaction area in front of the window, the projected figures turn their heads glancing at him.
If the pedestrian stops, they will move aside, parting enough to allow him a view onto what they’re looking at.

After the pedestrian has been in the interaction area for a period of time, the projected figures will turn to face him as one of the figures points him out. This sudden gesture of recognition, confrontation or accusation directed at him startles and singles out the viewer. The other projected figures look at him, silently questioning, scrutinizing, judging. As the pedestrian moves, the projected people they track him with their gazes. The pedestrian realizes that he is the one on display, he became the spectacle. The projected figures are waiting for what he will do next. If he gestures of moves, the projected audience will applaud, whisper to each other or take pictures with their virtual cameras.

The viewer quickly realizes that the installation is interactive, and tries waving his hands or doing other gestures to ‘activate’ it. To this the projected figures react by clapping, applauding the performance and clarifying their role of audience.

Installation at CAM Raleigh for Born Digital:

Viewer’s movement and expressions are mimicked by an animal’s head which is overlaid on the viewer’s reflection. The resulting effect invites inquiry into issues of self-awareness, empathy and non-verbal communication.

A different animal appears every time a person walks in front of the mirror. The animals represent species from across the spectrum of domestication, from wild predators to domesticated species to animals who have evolved to coexist with human settlements, while remaining ‘wild’.

The animal mimics the viewer’s facial expressions, interspersing them with its own independent ones. One feels compelled to in turn enact those animal expressions, lip licking and snarling, fully inhabiting the role, following while being followed.

The images above show how the person looking in the mirror sees the animal image: as overlaid on their own reflection. For others, seeing this interaction from a different perspective, the animal head is offset, as in the video below. (The pictures were taken with the animal being offset for the viewer).

Made using
openTSPS(opentsps.com/)
openFrameworks (openframeworks.cc/)
Unity3d (unity3d.com/)
Blender3d (blender.org/)

Background

Audience exposes the dramatic mechanisms of spectatorship, public gatherings and how they effect individual actions. The viewer is acknowledged and becomes a part of the ad-hock audience, and then the object of its scrutiny. He or she is inadvertently caught up in the social dynamic of curiosity and spectatorship, and invited to examine its nature.

The projected characters eventually transfer the attention from themselves to the viewer, spotlighting him and moving the staging area from under their feet to the sidewalk, upending the subjective roles of spectators, performers and participants. The audience’s attention invites the viewer to fill in the situation with the specifics of their individual circumstance, prompts introspective inquisitiveness. One’s anonymity in public space is called into question. The installation uses surveillance technology, and provokes the same privacy questions that its wide spread use does. Are we being singled out by the system? Perhaps we have unwittingly prompted suspicions by the way we walk or dress? The intelligence of these systems is often opaque and prompts suspicion, distrust and self-survaillance. The installation also relates different viewers to one another.

The installation, while being a literal display, simultaneously takes part in the pedestrian’s reality, the characters responding to the social space they help to create. At the same time it remains a metaphor that relies on the suspension of disbelief, and leads the viewers to examine their relationship to their social and physical environment.

Viewer’s movement and expressions are mimicked by an animal’s head which is overlaid on the viewer’s reflection. The resulting effect invites inquiry into issues of self-awareness, empathy and non-verbal communication.

A different animal appears every time a person walks in front of the mirror. The animals represent species from across the spectrum of domestication, from wild predators to domesticated species to animals who have evolved to coexist with human settlements, while remaining ‘wild’.

The animal mimics the viewer’s facial expressions, interspersing them with its own independent ones. One feels compelled to in turn enact those animal expressions, lip licking and snarling, fully inhabiting the role, following while being followed.

The images above show how the person looking in the mirror sees the animal image: as overlaid on their own reflection. For others, seeing this interaction from a different perspective, the animal head is offset, as in the video below. (The pictures were taken with the animal being offset for the viewer).

Made using
FaceTracker library from Jason Saragih (web.mac.com/jsaragih/FaceTracker/FaceTracker.html),
ofxFaceTracker addon by Kyle McDonald (github.com/kylemcdonald/ofxFaceTracker)
openFrameworks (openframeworks.cc/)
Unity3d (unity3d.com/)
Blender3d (blender.org/)

Background

Mirror neurons and on-verbal communication

Recently a neural mechanism has been discovered that explains how we get experiential insight of other minds.  Mirror neurons activate when we perform an action as well as when we watch someone perform an action, and thus have been implicated in imitation learning.  In addition to acting on the motor centers, mirror neurons have been theorized to play a role in ‘theory of mind’ concepts such as emotional recognition or contagion.  Emotional contagion is based on interpreting the emotional state of another being expressed through physical features.  Emotions (such as fear or sadness) have typical facial characteristics, and the mirror neurons theory would consider the neurons as ‘mapping’ the facial features of another being onto the respective areas in our own brain.  Evolutionary theory tells us that such direct mapping would be beneficial as it could help us perceive danger or threat more directly and quickly.  More complex evolutionary theories link mirror neurons to the appearance of cooperative behavior or emotions such as empathy and compassion.

The yet unanswered question is to what extent do mirror neurons might function between humans and animals, and do they function for more complex behaviors and emotions.  We do know that there is reciprocal behavior between different animal species, and a certain amount of mind modeling.  Studies of the predator-pray relationships tell us that survival in those roles depends on successful inserting of oneself into the mind of the hunter or the hunted, even to the point of mimicking its behavior.  “It was our ancestors’ skill at not only analyzing and imitating the nature of a given animal, but identifying with it, that enabled them to flourish in dangerous environments, both physically and psychically.” writes John Vaillant in The Tiger.  The ability to understand and communicate with other species is also apparent in our relationships with domesticated animals that rely on cooperation and emotional bonding.  Communication with animals in particular relies on a non-linguistic, body-based, instinctive and emotional aspect of expression. Social psychology studies have demonstrated that imitation and mimicry are pervasive, automatic, and facilitate empathy. The neural mirroring suggests that those mimicry exchanges are a bridge to inhabiting the other’s mind — and to the in-depth understanding of each other.

Mask: the provisional face

Throughout the world animal masks are used for their expressive and symbolic power – both ritually and in various theatre traditions. Often the mask is regarded as an instrument of revelation, giving form to the invisible. Many African tribes believe that the animal masks can help them communicate with the spirits who live in forests or open savannas. The neural mirroring mechanism suggests that these rituals might be far more than simply a symbolic performance — they are actually an embodied simulation of other creatures.

Symbolism is encoded into the designs of the masks to inform the spectators about the role and its dominant characteristics. These designs often exaggerate facial expressions.  For example The Senoufo people of the Ivory Coast represent tranquility by making masks with eyes half-shut and lines drawn near the mouth, and the Grebo of the Ivory Coast carve masks with round eyes to represent alertness and anger.

Nicola Savarese in “A Dictionary of Theatre Anthropology: The Secret Art of The Performer” discusses the ‘facial gestures’ expressive of emotional states that we share with animals. “The expression ‘to show one’s teeth’ is so rich in meaning that it has passed from physiology to proverb”, he writes and notes how we often intentionally ‘perform’ these facial gestures in order to communicate and thus turn our face into a mask: “The mask becomes a face and the face a mask. It is not the psychology of feelings but anatomy of forms which is being dealt with here.”

Mirrors, perception and the amalgamation of the physical and virtual

Part of my interest in this project is the combination of the virtual and physical world — inserting a layer of imagination into a world we know. The chain of causes and effects remains in place, although slightly augmented. The familiar is transformed into the uncanny, prompting us to see the mechanics of perception, interaction, and relationships with others anew.
“In a sense, mirrors are the best ‘virtual reality’ system that we can build,” said Marco Bertamini of the University of Liverpool. “The object ‘inside’ the mirror is virtual, but as far as our eyes are concerned it exists as much as any other object.” Dr. Bertamini and his colleagues have also studied what people believe about the nature of mirrors and mirror images, and have found nearly everybody to be shockingly off the mark.  (For example, no matter how far we stand from the mirror, our face in the mirror is always exactly half the size of our face: the mirror is always halfway between our physical selves and our projected selves in the virtual world inside the mirror, and so the captured image in the mirror is half our true size.)

To scientists, the simultaneous simplicity and complexity of mirrors make them powerful tools for exploring questions about perception and cognition in humans and other neuronally gifted species, and how the brain interprets and acts upon the great tides of sensory information from the external world. They are using mirrors to study how the brain decides what is self and what is other, how it judges distances and trajectories of objects, and how it reconstructs the richly three-dimensional quality of the outside world from what is essentially a two-dimensional snapshot taken by the retina’s flat sheet of receptor cells.
– from New York Times

Title

Konstanty Ciołkowski, a Soviet Russian rocket scientist, pioneer of cosmonautics, writer and philosopher used the phrase “all the universe is full of the lives of perfect creatures” in his “The Cosmic Philosophy” whose main idea is arriving at the absence of suffering for all the living beings in the Cosmos, for all the Universe.

It’s You is an interactive storefront-window projection that explores the mechanisms of public behaviors and the line between the real and constructed social actions. The installation’s content is partly created with the audience during a workshop that introduces techniques of creative storytelling, improvisational theater, animation and interaction design.

OVERVIEW

The installation is a rear projection on large storefront window. Projected human figures crowd around something that they obscure from the pedestrian’s view. Their backs are to the street, they are focused on something that’s happening in front of them. When the pedestrian stands behind them, as if to look over their shoulders, they step aside to allow him a view onto what they’re looking at. The pedestrian can now see part of the unfolding scene, and he obscures the view for the other pedestrians; he’s become part of the crowd.

During the workshop, with the help of local participants, we will develop the scenes that the virtual crowd is looking at. These hidden scenes will be suggestive of situations that are normally hidden from the public eye, both drawing
the viewer in to a story and provoking viewers to consider questions of social conduct and the behavior of individuals within systems of social restraint. The participants and I will brainstorm and reenact several different scenarios for those hidden scenes. We will capture these performances on the portable motion-capture system. These performances will be then applied to the figures in the projection: the 3d humans will perform the captured actions in front of our virtual crowd. While the installation is up, a different one of these hidden scenes will be playing every day.

The animated imagery is rear-projected onto a full-length window facing a public walkway. A rear-projection screen is placed inside the window or a film is applied to the glass surface.
The projections are of full-size CG characters whose behavior is scripted in a game engine to respond dynamically to the presence and behavior of the pedestrians.
An IR camera overlooks the sidewalk in front of the projection and the real-time video feed from it is analyzed with a computer vision software to determine the location and gestures of the pedestrians.
The characters are rendered white, with simplified polygonal structure. Each is an individual, with unique features and animation.
The light on the characters will be adjusted to be similar as the light at the site, to create a continuous transition from the real to virtual world. The reflections on the storefront glass further contribute to blending of physical reality with the virtual
one.

This sequence illustrates the behavior of the projected crowd when a pedestrian walks by on the sidewalk in front of the projection window.
When the pedestrian enters the interaction area in front of the window, the projected figures turn their heads glancing at him.
If the pedestrian stops, they will move aside, parting enough to allow him a view onto what they’re looking at.

After the pedestrian has been in the interaction area for a period of time, the projected figures will turn to face him as one of the figures points him out. This sudden gesture of recognition, confrontation or accusation directed at him startles and singles out the viewer. The other projected figures look at him, silently questioning, scrutinizing, judging. As the pedestrian moves, the projected people they track him with their gazes. The pedestrian realizes that he is the one on display, he became the spectacle. The projected figures are waiting for what he will do next. If he gestures of moves, the projected audience will applaud, whisper to each other or take pictures with their virtual cameras.

This sequence the projected crowd watching and reacting to the pedestrians’ performance.
The viewer quickly realizes that the installation is interactive, and tries waving his hands or doing other gestures to ‘activate’ it. To this the projected figures react by clapping, applauding the performance and clarifying their role of audience.

Documentation from an installation at CAM Raleigh:

CONCEPT

Audience exposes the dramatic mechanisms of spectatorship, public gatherings and how they effect individual actions. The viewer is acknowledged and becomes a part of the ad-hock audience, and then the object of its scrutiny. He or she is inadvertently caught up in the social dynamic of curiosity and spectatorship, and invited to examine its nature.
Public group gatherings are a part of participation in the civic life and religious rituals, they are forums for discussion, provide a sense of belonging and community, and enforce a common ethics and social responsibility.
A story unfolding on a social forum gains the added dimension of a symbolic, archetypal meaning.

The projected characters eventually transfer the attention from themselves to the viewer, spotlighting him and moving the staging area from under their feet to the sidewalk, upending the subjective roles of spectators, performers and participants. The audience’s attention invites the viewer to fill in the situation with the specifics of their individual circumstance, prompts introspective inquisitiveness.
One’s anonymity in public space is called into question. The installation uses surveillance technology, and provokes the same privacy questions that its wide spread use does. Are we being singled out by the system? Perhaps we have unwittingly prompted suspicions by the way we walk or dress? The intelligence of these systems is often opaque and prompts suspicion, distrust and self-survaillance. The installation also relates different viewers to one another. What will other people think if one is pointed out?

The installation, while being a literal display, simultaneously takes part in the pedestrian’s reality, the characters responding to the social space they help to create. At the same time it remains a metaphor that relies on the suspension of disbelief, and leads the viewers to examine their relationship to their social and physical environment.

WORKSHOP AND SITE SPECIFICITY

During the residency I will lead a workshop for the local participants during which we will create the content of the hidden scenes in the installation. The input from a local audience will not only grant them emotional
ownership and investment in the piece, but will also color the stories that will be created with the local concerns and culture, and resonate with viewers. The participants in the workshop will learn techniques of creative storytelling, improvisational performance, animation, motion capture and interaction design. Through the process of creating stories that resonate at the border of private and public spaces, the participants will get a chance to examine and share their ideas about community, social responsibility, and creative problem solving.

SITE

An ideal site for the installations would be an empty retail storefront with a full length window, in a neighborhood with a lot of foot traffic. The width of the window ideally would be around 12-14 feet, though the specific dimensions
are flexible, as well as the site conditions. I would like the installation to become a part of the neighborhood, I believe the people who encounter it often would invest the characters with their own ideas that would grow more complex over time and would become part of the history of these people’s lives.

TECHNICAL DETAILS

OVERVIEW
People on the sidewalk are monitored by an IR camera. In a custom software application, build with C++ library openFrameworks, each individual person is isolated and assigned a unique id for the duration of their interaction. Each persons’ position and gesture information is continually sent to the game application build with Unity3d game engine via OSC networking protocol. In Unity, an artificial intelligence system keeps track of all the pedestrians, chooses which person
to point at, and which projected character is going to be the pointer. Based on the interaction each projected person is assigned an appropriate behavior.

3D ARTWORK
The characters are modeled and animated in a 3d application Blender 3d. They are imported into the Unity3d game engine where they are controlled with scripts.

MOTION CAPTURE
During the workshop we will set up a portable motion- capture system that consists of 2 Kinect cameras. The depth data captured from them will be processed by a iPiSoft software to extract skeleton and animation data, and apply it to the virtual figures.

VIDEOTRACKING IN OPENFRAMEWORKS
To detect people I will be using the standard OpenCV blob extraction packaged with openFrameworks. Beyond the standard blob tracker, I will use a ‘PersonTracker’ which assigns persistent blob ids between frames, and dampens the movement of the centroids. In addition I will also run custom metrics to filter out false-positive detections, such as the glare from headlights that otherwise would register as people. Through blob detection I am given a good indication of the locations of people, but I will be also interested in how they are behaving. During each frame of video I run optical flow to detect motion. Within each person’s bounding rectangle I sum all the motion to determine if that individual is making sudden movements.

UNITY3D
The abstracted information for each individual is sent to Unity where the artificial intelligence and animation engine is contained. Communication between the two applications is handled by OSC messages containing person ids, location and gesture information. The Unity3d system works with an object oriented architecture, containing instances each projected person, pedestrians and relationships between them.

FABRICATION / SITE PREPARATION
There are two options for the video tracking set-up: using an IR sensitive camera and flooding the sidewalk with the IR light (mounting 4 IR lights at the top of our projection window), or using Kinect sensor, which is a 3d camera that is ‘light-invariant’: it doesn’t detect things based on how visible light reflects from them, but based on how close they are in physical space. The Kinect sensor would be preferable, but it has a smaller angle of view than a camera, so I would have to be place it very high up on the facade. The specifics of the site will determine which camera solution will be used. The rear-projection screen will be hung close to the glass, the projector and computer will be placed back in the room.
We will install the project at the site, and at the end of the week, the new scenes will be added. A different one will be triggered every day for the duration of the installation.

Surveyor Suite is a tool for exploration and re-imagining of the urban environment. The devices are used for performative research and public events. The resulting video, writing and images are collected online.

Surveyor is an iPhone app for use with a portable mini projector. A projected man surveys his surroundings and catalogues everything around him. He has binoculars and a log book, in which he intermittently writes his notes. The user of the app can direct the gaze of the man, examining the particular site he’s in. The virtual gaze of the surveyor directs the attention of the people around him, be it to other people or to their surroundings. Playful tool of engagement at a distance, it creates a connection between the app user and the people viewing the projection, as they become implicated in the piece. In this case the user is the puppeteer, utilizing the classic performance technique where the eyes of the performer move to where he wants to direct his, and therefore the spectators’ attention. The gaze functions like a telephoto lens, zooming in on a camera shot.

The projection is the entry point for re-interpreting and re-envisioning the reality of urban environment. Used in a public space, the Surveyor becomes a public engagement tool that inspires personal involvement and facilitates public discussion. Activated by the imagination of the viewers, it is a catalyst that lets us see our surroundings anew, wonder what’s behind the walls and reevaluate things by inviting speculation and questioning through storytelling and imaginative thinking.

Work in progress:

Surveyor Suite is a part of Amateur Human project

Made using
Unity3d (unity3d.com/)
Blender3d (blender.org/)

Background

Lawrence Weschler talks about the ideas of Nicholas of Cusa (an early Renaissance mathematician and mystic):

To gain knowledge of God, one should just catalog everything — books, rocks, flowers, human emotions — so that by the end you have cataloged all of God’s creations. So Cusa, who is a mathematician, says: “Well, I suppose that’s a bit like an n-sided polygon inside of a circle.” In other words, you take a triangle inside of a circle, and you keep adding sides to it, and the more sides you add the closer it comes to being a circle. And yet, at the same time, you keep getting further away, because a circle only has one “side,” one line, and here you’ve got a million lines and angles. Cusa was the one to come up with this concept that at a certain point, you have to make a leap of faith–from the n-sided polygon to the circle, say. And that leap of faith is accomplished through grace, which is to say, for free.

This is similar to the theoretical physics’ quest for the “Theory of Everything”. As Margaret Wertheim points out, science is founded on the belief that things are comprehensible and that by the ingenuity of our minds and the probing of ever more subtle instruments we will ultimately come to know It All. But is the All inherently knowable?

In the quantum teleportation experiments physicists ‘teleport’ a photon — or replicate all the physical properties of a photon in a photon in another location. Borges writes about the map that is so detailed that is the exact size of the territory. In his story,the cartographers realize how pointless it became and the effort is abandoned, reminding us of the function of abstraction.

Shadowplay is a mobile game for the use with a portable projector. In this game, an animated image of a wolf is projected from the device. The user, using multitouch gestures, can direct the animal’s movement to interact with the environment it’s projected onto. For the ISEA exhibition a virtual wolf habitat is mapped onto the area of Albuquerque. The devices are distributed to participants, who are encouraged to explore this habitat, ‘puppeteering’ the wolf in the public. The game design will be shaped by the research done during the residency using the NMWA wolf data collection. Resources used by wolves such as water, food and shelter are scattered around the ‘habitat’ area using geolocation markers. The wolves can also encounter traps in their habitat. During the workshop, the participants learn about the wolves lives, environment and the history of their relationship with people.

A short video showing work in progress animation and touch gesture control (the password is ‘shadowplay’):

Animal myths and folktales have regulated human behavior towards animals for centuries. The myths formalize and generalize the culture’s relationship to them. They often help us to respect and understand the animal’s habits as well as encouraging us to keep our distance. The wolf — real and imagined — in particular has held deep meaning for humans. Revered and vilified, today it is often the victim of our collective imagination. The old vestiges of myths can lead to misunderstanding and mistreatment of wildlife: our stories and attitudes are in need of updating to reflect wolf’s and our place in the changing ecology.
Shadowplay aims to inspire a new perspective by giving the participants a way to not only familiarize themselves with facts about wolfs, but more importantly identify with the wolf, create an empathetic emotional connection to it. The participants create new kind of ‘folktales’ based on what they learned and what they imagine, of how the wolves interact with their environment and with people. They are specifically encouraged to pick apart the assumptions that the old myths left in popular culture and to see the animals as neither demonized or glorified. All these stories, writings and audio recordings as well as images and videos from the city explorations, are collected online and shown in the exhibition, contributing to collective new wolf ‘mythology’.

Play is the domain of innovation as well as imagination. As they explore with their projections, the participants are encouraged to point out the problems and propose solutions to managing meeting ours and wolves’ needs.

As they project on the city walls the participants become performers, drawing attention and potentially participation of others, turning their play into a public address. The sight of a wild animal is a rare and wondrous experience. It inspired the ancient humans to cover their walls with drawings of wild animals, and no less impresses us today. A projection of an animated animal in an urban setting carries on a part of that wonder and inspiration, and the reminder that these animals might become mere shadows in our memory if we do not protect them.

In this project one looks into a public mirror, and in that moment, one sees himself as other. The installation introduces a transposition of identities and expressions, enacting a potential of inhabiting wildness.

A mirror along a public walkway is a reflective surface in more than one sense: one can see one’s own image combined with that of an animal. Viewer’s movement and expressions are matched by an animal’s head. This image is also projected large-scale on a side of a building. The resulting effect, at times dream-like, comical or profound, invites inquiry into issues of self-awareness, empathy and communication, in addition to creating a connection to the natural world.

The images above show how the person looking in the mirror sees the animal image: as overlaid on their own reflection. For others, seeing this interaction from a different perspective, the animal head is offset, as in the video below. (The pictures were taken with the animal being offset for the viewer).

The sight of a wild animal is a rare and wondrous experience. It inspired the ancient humans to cover their walls with drawings of wild animals, and no less impresses us today. A projection of an animated animal in an urban setting carries on a part of that wonder and inspiration, and creates an emotional and personal connection to the natural world.

A different animal appears every time a person enters in front of the mirror. Several different models of animals are created and animated. I will specifically focus on wild species that used to roam what is now Brooklyn, and choose species that bear a lot of weight in our imaginations: for example wolves or bears. Those species conincidentally are also often endangered. This installation focuses our attention on them, existing as phantoms in our imaginations while disappearing from physical reality. As the projection on the building mimics the viewers, they become performers, drawing attention and participation of others, turning their play into a public address.

This installation is an amalgamation of the physical and virtaul that allows us to inhabit another being — a wild animal.
The animal mimics the viewer’s facial expressions, interspersing them with its own independent ones. One feels compelled to in turn mimick those animal expressions: enact the lip licking and snarling, fully inhabiting the role, following as well as being followed.

Background:

Mirror neurons are neurons that fire both when an animal acts and when the animal observes the same action performed by another. Thus, the neuron “mirrors” the behavior of the other. Researchers have linked the mirror neuron system to communication, understanding other’s goals and intentions, and empathy. These neurons can not only help simulate other’s behavior but also help us understand it through partly experiencing it. Our sensory system is always in a sense blending with those of others. Neuroscientist V.S. Ramachandran writes that the only thing that separates us from each other is a small subset of neural circuits in our frontal lobes interacting with mirror neurons.

Facial expressions are one of the most intuitive and inate channels of communication between us and animals. We often mimic our domesticated animals’ expressions to interact with them. Both humans and domesticated animals have evolved to communicate with each other through these ‘conversations’ of mimicry.

Throughout the world masks are used for their expressive and symbolic power – both ritually and in various theatre traditions. Often the mask is regarded as an instrument of revelation, giving form to the invisible (and often sacred). Many African masks represent animals, and some of the tribes believe that the animal masks can help them communicate with the spirits who live in forests or open savannas.

Cheshire Cat

Alice encounters it in Wonderland, where it engages Alice in amusing but sometimes vexing conversation. The cat sometimes raises philosophical points that annoy or baffle Alice. One of its distinguishing features is that from time to time it disappears, the last thing to be seen being its grin.

An exhibit called The Cheshire Cat at the Exploratorium in San Francisco, created by Bob Miller in 1978, features a mirrored eyepiece that allows visitors to look at a picture of the Cheshire Cat’s face with one eye, while the other eye sees a reflection of a white screen to the side. When the visitor waves a hand in across the white screen, the cat image starts to disappear. If the visitor focuses on the cat’s smile while doing this, the smile will remain while the cat disappears. The general phenomenon of a moving stimulus presented to one eye causing a static image to disappear from the other eye is called the Cheshire Cat effect, named after this exhibit. The effect is part of a broader visual phenomenon called binocular rivalry.

Mirror

To scientists, the simultaneous simplicity and complexity of mirrors make them powerful tools for exploring questions about perception and cognition in humans and other neuronally gifted species, and how the brain interprets and acts upon the great tides of sensory information from the external world. They are using mirrors to study how the brain decides what is self and what is other, how it judges distances and trajectories of objects, and how it reconstructs the richly three-dimensional quality of the outside world from what is essentially a two-dimensional snapshot taken by the retina’s flat sheet of receptor cells.

– article in New York Times

Szarkowski, the curator of MOMA photography show “Mirrors and Windows” recognized that photography had become the way we now perceived reality itself. It didn’t capture reality so much as reflect it back in some interesting and perplexing ways, as mirrors do.

Mirror neurons

Mirror neurons are neurons that fire both when an animal acts and when the animal observes the same action performed by another. Thus, the neuron “mirrors” the behavior of the other, as though the observer were itself acting. Such neurons have been directly observed in primate and other species including birds. In humans, brain activity consistent with that of mirror neurons has been found in the premotor cortex, the supplementary motor area, the primary somatosensory cortex and the inferior parietal cortex.

The function of the mirror system is a subject of much speculation. Many studies link mirror neurons to language abilities, learning new skills by imitation, and understanding goals and intentions thus contributing to theory of mind skills. Researchers have argued that the mirror neuron system is involved in empathy: A large number of experiments using functional MRI, electroencephalography (EEG) and magnetoencephalography (MEG) have shown that certain brain regions (in particular the anterior insula, anterior cingulate cortex, and inferior frontal cortex) are active when people experience an emotion (disgust, happiness, pain, etc.) and when they see another person experiencing an emotion. Recent evidence suggests that there may be mirror neurons for pain, disgust, facial expression—perhaps for all outwardly visible expression of emotions.

Neuroscientist V.S. Ramachandran writes: “… I also speculated that these neurons can not only help simulate other people’s behavior but can be turned “inward”—as it were—to create second-order representations or meta-representations of your own earlier brain processes. This could be the neural basis of introspection, and of the reciprocity of self awareness and other awareness. There is obviously a chicken-or-egg question here as to which evolved first, but… The main point is that the two co-evolved, mutually enriching each other to create the mature representation of self that characterizes modern humans.”

Ramachandran describes how the mirror neuron signals are inhibited (despite their activity we don’t actually mimic people), and goes on to note: “despite all the pride that your self takes in its individuality and privacy, the only thing that separates you from me is a small subset of neural circuits in your frontal lobes interacting with mirror neurons. Damage these and you “lose your identity”—your sensory system starts blending with those of others.”

Mask: the provisional face

Throughout the world masks are used for their expressive and symbolic power – both ritually and in various theatre traditions. Often the mask is regarded as an instrument of revelation, giving form to the invisible (and often sacred). This is accomplished for example by linking the mask to an ancestral presence, and thus bringing the past into the present. Many African masks represent animals, and some of the tribes believe that the animal masks can help them communicate with the spirits who live in forests or open savannas.

Symbolism is encoded into the designs of the masks to inform the spectators about the role and its dominant characteristics. For example The Senoufo people of the Ivory Coast represent tranquility by making masks with eyes half-shut and lines drawn near the mouth. The Temne of Sierra Leone use masks with small eyes and mouths to represent humility and humbleness. They represent wisdom by making bulging forehead. Other masks that have exaggerated long faces and broad foreheads symbolize the soberness of one’s duty that comes with power. The Grebo of the Ivory Coast carve masks with round eyes to represent alertness and anger, with the straight nose to represent unwillingness to retreat.

Nicola Savarese in “A Dictionary of Theatre Anthropology: The Secret Art of The Performer” discusses the ‘facial gestures’ expressive of emotional states that we share with animals (“the expression ‘to show one’s teeth’ is so rich in meaning that it has passed from physiology to proverb”) and notes how we often intentionally ‘perform’ these facial gestures in order to communicate and thus turn our face into a mask: “The mask becomes a face and the face a mask. It is not the psychology of feelings but anatomy of forms which is being dealt with here.”
Savarese describes a mask as a technique to give the face an ‘extra-daily’ dimension. However it is often considered as something artificial the represses the performer, since he or she looses the ability to use the rich facial musculature and is stuck with a ‘dead’ face that calls to mind a decapitation.

The mask that is animated and has its own apparent agency that might be conflicting with the mask wearer’s creates a different dynamic.

Amalgamation of the physical and virtual

Part of my interest in this project is the combination of the virtual and physical world — inserting a layer of imagination into a world we know and make it follow the rules we know, though with a slight change. The chain of causes and effects remains in place, although slightly augmented. The familiar is transformed into the uncanny, prompting us to see the mechanics of perception, interaction and relationships with others anew.
“In a sense, mirrors are the best ‘virtual reality’ system that we can build,” said Marco Bertamini of the University of Liverpool. “The object ‘inside’ the mirror is virtual, but as far as our eyes are concerned it exists as much as any other object.” Dr. Bertamini and his colleagues have also studied what people believe about the nature of mirrors and mirror images, and have found nearly everybody, even students of physics and math, to be shockingly off the mark.
For example, no matter how far we stand from the mirror, our face in the mirror is always exactly half the size of our face: the mirror is always halfway between our physical selves and our projected selves in the virtual world inside the mirror, and so the captured image in the mirror is half our true size.

Title

Konstanty Ciołkowski, a Soviet Russian rocket scientist, pioneer of cosmonautics, writer and philosopher used the phrase “all the universe is full of the lives of perfect creatures” in his “The Cosmic Philosophy” whose main idea is arriving at the absence of suffering for all the living beings in the Cosmos, for all the Universe.

Crossing Paths is an iPhone app for use with a portable mini projector. It features a 3d animal of one of the urban dweller species (such as red fox, falcon or coyote). The animal’s actions can be directed by the user of the app to engage with the environment it’s projected onto.

PARTICIPATION

The devices (the projector with the iPod touch) are distributed to teams of participants, who are encouraged to take them around the city, exploring and re-imagining it as a habitat for their projected creature. The participants receive information about their creature and what urban habitats might support it. Each team documents their exploration and begins to create new stories around their animal based on what they learned and what they imagine, of how the animals interact with their environment, and with people. They are specifically encouraged to pick apart the assumptions that the old myths left in popular culture and to see the animals as neither demonized or glorified. The resulting videos, images and texts are uploaded to the website, where they serve as a base for a new ‘mythology’ of the animals.

Optionally, the users may mark their projected animals’ paths as geolocation points through their app. Each user can trace the animal paths imagined by the others.

A short video showing work in progress animation and touch gesture control (the password is ‘shadowplay’):

Myths and folktales regulate our behavior towards animals: they help us to respect and understand the animal habits but also to keep our distance. As our environment is rapidly changing the old vestiges of myths often lead to misunderstanding and mistreatment of wildlife: our stories and attitudes are in need of updating.

The participants become performers as they project on the city walls, drawing attention and participation of others, turning their play into a public address.

The sight of wild animals inspired ancient humans to cover their walls with drawings of wild animals, and today we are no less impressed by a sighting of a falcon or a fox. A projection of an animated animal in an urban setting carries on a part of that wonder and inspiration, and creates an emotional connection to the natural world. Once we are aware that we share our habitat withe other species, it’s easy to imagine their paths crossing our paths. And once we have internalized the idea of their presence among us, we will naturally account for them in our behavior, our policy, our urban planning, and our cultural stories.

Surveyor Suite is a tool for exploration and re-imagining of the urban environment. The devices are distributed among a group of participants for user tests, performative research and public events. The resulting re-visions (video, writing and images) are collected online.

Surveyor Suite is an iPhone app for use with a portable mini projector. It has 3 modes, each featuring a human figure whose actions can be directed by the user to engage with the environment it’s projected onto.  The simple actions of each figure are the entry point for re-interpreting and re-envisioning the reality of urban environment. Used in a public space, the Suite becomes a public engagement tool that inspires personal involvement and facilitates public discussion. Activated by the imagination of the viewers, it is a catalyst that lets us see our surroundings anew, wonder what’s behind the walls and reevaluate things by inviting speculation and questioning through storytelling and imaginative thinking.

Surveyor

In this app a man surveys his surroundings and catalogues everything around him. He has binoculars and often looks through them. He also has a log book, in which he intermittently writes his notes. The user of the app can direct the gaze of the man, examining the particular site he’s in. The virtual gaze of the surveyor directs the attention of the people around him, be it to other people or to their surroundings. Playful tool of engagement at a distance, it instantly creates a connection between the app user and the people viewing the projection, as they become implicated in the piece. In this case the user is the puppeteer, utilizing the classic performance technique where the eyes of the performer move to where he wants to direct his, and therefore the spectators’ attention. The gaze functions like a telephoto lens, zooming in on a camera shot.

Work in progress:

Examiner

Here a woman examines the surface she’s projected on. She puts her ear to the wall, listening in for what might be happening on the other side. She inspects the surface with her hands, prying into cracks and openings, often with her back to the viewer. The user of the app can direct her attention, leading her to examine different parts of the surface in an appropriate way.

Prospector

Prospector is looking for valuable things. He is willing to entertain a notion of that every thing might contain value, and its a matter of the right approach to get at it. He invites reflection on our value system and our speculation on what is truly irreplaceable and what should be valued highly. He attempts to put together a ‘true’ value system. The user of the app can direct the prospector’s attention, and invest it with his own speculative thoughts. He is like an archeologist from the future, considering our artifacts, spaces and designs for the first time and removed from any context.

The projections can installed indoors. Here the surveyor is perched atop a board leaning on a bookshelf. He has binoculars and a log book. He observes and notes. Sometimes he tries to get off his perch, but his attempts are not successful, so he remains contained, surveying.

The apps with the projectors are distributed among a group of participants. Each participant keeps a log of thoughts and ideas inspired by the interaction of the projection with the environment, which are then collected online, creating a shared imagined space of a shared physical space. Each story may be accompanied by the photos of the site with the projection and with a google maps location. The stories embody the reflections on engagement with the participant’s surroundings, curiosity and personal connections to it. The careful considerations of the details of the physical space as observed from a specific point of view activates it in imagination and memory in unexpected ways.

Surveyor Suite is a part of Amateur Human project

Background:
Lawrence Weschler talks about the ideas of Nicholas of Cusa (an early Renaissance mathematician and mystic):

To gain knowledge of God, one should just catalog everything — books, rocks, flowers, human emotions — so that by the end you have cataloged all of God’s creations. So Cusa, who is a mathematician, says: “Well, I suppose that’s a bit like an n-sided polygon inside of a circle.” In other words, you take a triangle inside of a circle, and you keep adding sides to it, and the more sides you add the closer it comes to being a circle. And yet, at the same time, you keep getting further away, because a circle only has one “side,” one line, and here you’ve got a million lines and angles. Cusa was the one to come up with this concept that at a certain point, you have to make a leap of faith–from the n-sided polygon to the circle, say. And that leap of faith is accomplished through grace, which is to say, for free.

This is similar to the theoretical physics’ quest for the “Theory of Everything”. As Margaret Wertheim points out, science is founded on the belief that things are comprehensible and that by the ingenuity of our minds and the probing of ever more subtle instruments we will ultimately come to know It All. But is the All inherently knowable? Will there always be things we don’t know?

We can imagine a simulation that approximates the real thing to a degree that it’s practically indistinguishable from it. In the quantum teleportation experiments physicists ‘teleport’ a photon — or replicate all the physical properties of a photon in a photon in another location. Borges writes about the map that is so detailed that is the exact size of the territory. In his story,the cartographers realize how pointless it became and the effort is abandoned, reminding us of the function of abstraction.

For the most part, simulations, as all models, are abstractions: they don’t try to represent the reality exactly, but very selectively model a few elements, and impose on them the rules governing the behavior of these elements to create a dramatic tension. Just as architectural or industrial product models are stripped of their non-essential details, they are generalized to expose the aspect to be studied, the particular ‘thingness’ of the thing. I like this idea of a model as a ‘pure’ form of the essence of the thing, that can be secondarily invested with details. Read as such on a symbolic or abstract level, simulation functions very differently from the ‘representational’ simulations, and is better suited to the pursuit of meaning, seeing larger patterns and connections: seeing the forest rather than the trees.

Capacity (Light Blue)
-horizontal raised surface a foot from the floor 7′ x 3.9′
-projection down from the ceiling
-16×9 resolution projector ( 2500 lumen or higher, preferably DLP and good color resolution)
-Mac Pro tower
-small prints on the walls

Pornographic Pursuit (Red)
-front projection on the wall
-projector mounted on the ceiling (4:3 resolution, 2500 lumen or higher, preferably DLP)
-Mac Mini
-sensor matte 3 x 4 ft on the floor in front of the projection, connected to the computer with a cable. The matte should be around 6 ft away from the wall. Sensor supplied by Karolina.

Stability (Orange)
-front projection on the wall
-projector mounted on the ceiling (4:3 resolution, 2500 lumen or higher, preferably DLP)
-waist-high pedestal in front
-sensor cube (supplied by Karolina)
-Mac Mini
-maintenance: requires daily changing and overnight recharging 4 AA batteries in the sensor.

All the Universe is Full of the Lives of Perfect Creatures (Dark Blue)
-vertically mounted lcd monitor (approximately 23″ diagonal)
-wall mount for the monitor
-Mac mini
-Logitech C910 webcam (or similar wide angle) (supplied by Karolina)
-half-silvered mirror and a box housing it and covering everything else (size depends on the size of the monitor)

Surveyor (Green)
-Optoma PK301 Pico projetor
-mounting arm
(such as this: http://www.amazon.com/Manfrotto-244-Variable-Friction-Bracket/dp/B00193W3Z2/ref=sr_1_2?ie=UTF8&qid=1322683369&sr=8-2
http://www.amazon.com/Manfrotto-035RL-Super-Clamp-Standard/dp/B0018LQVIA/ref=pd_bxgy_p_img_b)
-lcd monitor and a dvd player or computer playing back a documentation video from Surveyor’s use in the city

Pedestal (Light Green)
-Optoma PK301 Pico projetor
-mounting arm
(such as this: http://www.amazon.com/Manfrotto-244-Variable-Friction-Bracket/dp/B00193W3Z2/ref=sr_1_2?ie=UTF8&qid=1322683369&sr=8-2
http://www.amazon.com/Manfrotto-035RL-Super-Clamp-Standard/dp/B0018LQVIA/ref=pd_bxgy_p_img_b)
-shelf
-stool

+ Storefront Projection
Sniff or It’s You

-rear projection (about 3500 lumen or more, DLP projector)
-rear projection screen or other projection surface (window glass can be frosted for example)
-Mac mini
-speakers and audio interface to computer
-Kinect camera, mount and usb 2.0 extension

Amateur Human is a multidisciplinary exploration of the issues of consumption, environmental ethics and social responsibility. This multi-part project encompasses designing accessories that personalize and make public our relationship with the environment, investigation and documentation of the process of bringing such devices to market, and distribution of the accessories to the users, documenting and exploring facets of the consumption-related issues.

This project is exploring design’s potential, at the heart of consumer culture, to be subverted for other ends, such as critical and artistic exploration. The accessories are not meant for mass-production, but are conceptual design proposals, prototypes that ask what if. Their form is slightly absurd and humorous, making them non-didactic and inviting wide-ranging interpretations. The objects often rely on the voyeuristic transgression of private space turned public.

Another aspect of the project is the exploration of the user-end experience of the accessory. Each user’s perspective throws a little light on the entangled issues of environmental stewardship, social responsibility, and our desires, concerns and responsibilities regarding sustainability issues. These perspectives are captured in a long-form documentary, and in the interviews presented online.

Amateur Human received an Emerging Fields grant from Creative Capital.
Project’s website

CAM Raleigh

Several of my installations are included in Born Digital exhibition at Contemporary Art Museum in Raleigh which opens January 28th, 2012, and is up until April 30th.

CAM Raleigh’s newest feature exhibition, Born Digital, invites visitors to physically explore digital culture and movement-driven artwork—to exercise their creativity and act on their curiosity. Opening January 28, 2012, the exhibition showcases the contemporary, visitor-dependent art of 12 national and international pioneers of digital and new media art.
Featured Designers and Artists
Born Digital contributors include: Advanced Media Lab, Jacob Ciocci, R. Luke DuBois, Channel TWo, Brent Green, Ajay Kurian, LoVid, Cole Pierce, Dennis Rosenfeld, Daniel Rozin, Scenocosme, and Karolina Sobecka.

Yale Art School

Frank Pichel and I will be doing a artist presentation at Yale Art School January 24, 2012.
The presentation is titled ‘Context’ and we’ll discuss how consideration of the audience impacts the work.

Nuit Blanche New York: Bring to Light

I will participate in this year’s Nuit Blanche New York event: Bring to Light October 1st, 2011, 6pm to midnight. My two installation will be at storefronts at the opposites sides of Franklin Ave at Greenpoint Ave.

Bring to Light is a free nighttime public festival of art in New York City that takes place simultaneously with “nuit blanche” events in cities around the world. Inviting emerging and established artists to make site-specific installations of light, sound, performance and projection art, the event creates an immersive spectacle for thousands of visitors to re-imagine public space and civic life. Bring to Light will transform streets, parks and the industrial waterfront of Greenpoint, Brooklyn set against dramatic views of the Manhattan skyline.

Special Projects Award 2011

I received the 2011 Special Project Award from Princess Grace Foundation for the further development of Amateur Human.

NYFA Fellowship 2011

I’m happy to announce that I received this year’s New York Foundation for the Arts Fellowship in Digital/Electronic Arts! This year the Fellowships were given in the following categories: Crafts/Sculpture, Digital/Electronic Arts, Nonfiction Literature, Poetry, Printmaking/Drawing/Book Arts. For the list of all the fellows in all the categories, the finalists and panelists, see NYFA’s website.

Media Archeology 2011

“Sniff” was invited to the Aurora Picture Show, which this year presents the 8th annual Media Archeology Festival Thursday, September 15 through Saturday, September 17, 2011 in Houston. Taking place at venues across the city, including The Orange Show and the Museum District, the festival is presented in collaboration with the University of Houston Cynthia Woods Mitchell Center for the Arts. This year, the festival explores the ideas of games and play from the past and present.

Interaccion I/O/I

“Sniff” is up in Barcelona until the end of August 2011, part of the Interaccion I/O/I, an exhibition at the DHUB Museum. A little bit about the exhibition:

I/O/I. The senses of machines (Interaction Laboratory) proposes an experimental and educational look into the relationship between man and machine at a time when new generation interfaces include natural interactive features and social behaviors. It is an exploration of new areas where interaction design is developed and generates new needs, disciplines and experiences.

Amateur Human is a multidisciplinary exploration of the issues of consumption, environmental ethics and social responsibility. This multi-part project encompasses designing accessories that personalize and make public our relationship with the environment, investigation and documentation of the process of bringing such devices to market, and distribution of the accessories to the users, documenting and exploring facets of the consumption-related issues.

This project is exploring design’s potential, at the heart of consumer culture, to be subverted for other ends, such as critical and artistic exploration. The accessories are not meant for mass-production, but are conceptual design proposals, prototypes that ask what if. Their form is slightly absurd and humorous, making them non-didactic and inviting wide-ranging interpretations.

Another aspect of the project is the exploration of the user-end experience of the accessory. Each user’s perspective throws a little light on the entangled issues of environmental stewardship, social responsibility, and our desires, concerns and responsibilities regarding sustainability issues. These perspectives are captured in a long-form documentary, and in the interviews presented online.

Amateur Human received an Emerging Fields grant from Creative Capital.
Project’s website

Puff is the first of the accessories, and 3 more are in the works.

Human agency is the capacity for human beings to make choices and to impose those choices on the world. It is normally contrasted to natural forces, which are causes involving only unthinking deterministic processes.
The capacity of a human to act as an agent invests a moral component into a given situation. Human agency entitles the observer to ask should this have occurred? in a way that would be nonsensical in circumstances lacking human decisions-makers. (source: Wikipedia)

Computers afford us the ability to run processes and execute rule-based symbolic manipulation. In computer ‘games’ or ‘simulations’ we can represent how real and imagined systems work.

Interactive installations too are logical systems which create relationships between a few interdependent elements, one of which is often the viewer.

The workings of these rule-based representations can be partly exposed in a visual ‘debug’ mode. While the game scene shows the seamlessly integrated effects of the rules implemented in the system, the visual debug mode is designed to shows the effects of each of the rules, to literally ‘picture it’ and see it work. In this view they can be easily scrutinized to improve, detect anomalies and assess impact, but more interestingly they also form a raw and unmediated snapshot of the ‘conflicts’ or relationships from which the project is built.

While the outward expression is communicated to the audience as a narrative metaphor, the internal symbolism of the debug lines can be read on a different level. Even without the knowledge of what the debug symbols signify, they form an impression of bearing a meaning encoded in a visual language, behind which are rules of logic. The abstract debug symbols, interposed on the renderings of elements whose behavior is driven by the debugged rules, imparts to those elements the weight of their rule-bound necessity. At their meeting, the two pictorial spaces inform and enrich one another, the illusion is both exposed and justified.

Even though these simulated systems tend towards clean abstractions and explicit rules, they gain ‘noise’ as they are implemented and expressed in visual terms. The appearance of the debug symbols is subservient to their function, but it is arbitrary as far as it serves the preferences of perceptual analysis (and as far as being within the vocabulary of the software tool). Their color is meant to stand out from the other colors, their size is meant to be seen from a perspective at which the given phenomenon can be observed, their form is chosen to communicate the position, direction, movement or other property of the element. Thus their arrangement and the resulting composition are the pictorial translation of the stakes portrayed in the work.

Alex Galloway writes about games being ‘the occult logic of software’.

Code hides itself in the very act of consummating its own expression. A game expends itself in the very act of its being played. And so the game retreats from its own essence. But only in such a way as to be more true than the essence could ever be.

Eddo Stern writes about the ‘artifacts’:

Most fantasy game designers would regard visible signs of any technological underpinnings as unwanted anachronisms that would threaten the constitution of the immersive fantasy they are attempting to construct. The resulting by-products of this problem can be found in the designers’ introduction of metaphors that function to assimilate unwanted technological residues into the narrative diegesis.

•A note on “Artifacts”
I am borrowing the term artifact from computer science where the term is used in reference to undesired cosmetic disturbances such as jagged edges or dirty patches in an image file (common in compressed digital video or jpeg images for example), excess noise or hiss in a sound stream, or unpredictable ASCII characters in a text file. Artifacts differ from bugs, which are usually caused by programming mistakes; artifacts don’t prevent functionality per se, but cause an unperfected aesthetic disturbance.

The images in this project show the visual ‘debug’ mode of the project Forth in which deterministic forces are pitched against the questionably ‘willful’ efforts of the people.

Forth is a public art installation commissioned by the University of Central Florida for the Institute for Simulation and Training.

Forth employs computer simulation techniques, interactivity and real time audio synthesis to generate a dynamically changing ocean scene. The seascape projected on a large curved screen is subject to the local weather conditions: the wave height and the atmospheric conditions reflect the weather sampled in Orlando and at the Florida coast. Small row boats containing groups of people advance across the expanse of open water, and narratives emerge within each vessel as the rowers try to stay afloat and on course in turbulent waters. A second screen in the lobby displays an underwater view of the scene, offering an unexpected perspective. Immersive sound heightens the richness of the experience, responding to the weather, and to the flow of visitors through the lobby.
The viewers in the lobby are tracked by a camera and are represented in the projection as paper boats floating on the ocean. The rowers may pick up these paper boats — bridging again the real and virtual worlds.

Software development for Forth was led by James George.  Sound design was created by Michael McCrea.

Project’s site

A 13-min video documenting the installation:

And a few captured screenshot movies of the projection content:

The movie above shows a few of the ‘features’ of the project:

• the weather changes according to the weather conditions in Orlando
• the waive-height of the ocean changes according to the weather conditions at the Florida coast and in Orlando
• color palettes change every week, with a small variation every day, transitioning between different color treatments for daytime and nighttime.
• the direction of the boats changes slowly over the course of the year
• the distribution of the boats varies all the time and is designed to create interesting compositions
• people in boats lean to stay upright, compensating for the tilt of the boat
• people’s rowing is also adjusted to the weather conditions
• people walking in front of the projection in the lobby are represented as paper boats floating on the ocean in front of the virtual camera
• the paper boats left on the ocean by the people present in the lobby can be picked up by the people in boats
• two of the ‘directorial gestures’ are shown in the demo: the ‘no rowing gesture’: people in the boats stop rowing and look at the viewer and the ‘stray boat gesture’: one of the boats turns and starts going in a different direction than all the other boats.
• the underwater scene is shown on an LCD monitor opposite of the projection