A projection in the corner of two adjoining walls is activated by viewer interaction.
The interface to activate it is a big heavy rock.
There are instructions on the wall for various ways of picking up the rock.
But the rock is too heavy to be picked up by most people — they can barely budge it.

If someone exerts sustained effort in lifting the weight of the rock, the sun starts rising.

It’s You is an interactive storefront-window projection that explores the mechanisms of public behaviors and the line between the real and constructed social actions.

The installation is a rear projection on large storefront window. Human figures crowd around something that they obscure from the pedestrian’s view. When the pedestrian stands behind them, as if to look over their shoulders, they step aside to allow him a view onto what they’re looking at. The pedestrian can now see part of the unfolding scene, and he obscures the view for the other pedestrians; he’s become part of the crowd.

When the pedestrian enters the interaction area in front of the window, the projected figures turn their heads glancing at him.
If the pedestrian stops, they will move aside, parting enough to allow him a view onto what they’re looking at.

After the pedestrian has been in the interaction area for a period of time, the projected figures will turn their attention to him. The viewer becomes the performer. If he does something to entertain his viewers, the projected figures will react by clapping, applauding the performance and clarifying their role of audience.

Installation at CAM Raleigh for Born Digital:

Made using
openTSPS(opentsps.com/)
openFrameworks (openframeworks.cc/)
Unity3d (unity3d.com/)
Blender3d (blender.org/)

Background

It’s You exposes the dramatic mechanisms of spectatorship, public gatherings and how they effect individual actions. The viewer is acknowledged and becomes a part of the ad-hock audience, and then the object of its scrutiny. He or she is inadvertently caught up in the social dynamic of curiosity and spectatorship, and invited to examine its nature.

The projected characters eventually transfer the attention from themselves to the viewer, spotlighting him and moving the staging area from under their feet to the sidewalk, upending the subjective roles of spectators, performers and participants. The audience’s attention invites the viewer to fill in the situation with the specifics of their individual circumstance, prompts introspective inquisitiveness. One’s anonymity in public space is called into question. The installation uses surveillance technology, and provokes the same privacy questions that its wide spread use does. Are we being singled out by the system? Perhaps we have unwittingly prompted suspicions by the way we walk or dress? The intelligence of these systems is often opaque and prompts suspicion, distrust and self-survaillance. The installation also relates different viewers to one another.

The installation, while being a literal display, simultaneously takes part in the pedestrian’s reality, the characters responding to the social space they help to create. At the same time it remains a metaphor that relies on the suspension of disbelief, and leads the viewers to examine their relationship to their social and physical environment.

Viewer’s movement and expressions are mimicked by an animal’s head which is overlaid on the viewer’s reflection. The resulting effect invites inquiry into issues of self-awareness, empathy and non-verbal communication.

A different animal appears every time a person walks in front of the mirror. The animals represent species from across the spectrum of domestication, from wild predators to domesticated species to animals who have evolved to coexist with human settlements, while remaining ‘wild’.

The animal mimics the viewer’s facial expressions, interspersing them with its own independent ones. One feels compelled to in turn enact those animal expressions, lip licking and snarling, fully inhabiting the role, following while being followed.

The images above show how the person looking in the mirror sees the animal image: as overlaid on their own reflection. For others, seeing this interaction from a different perspective, the animal head is offset, as in the video below. (The pictures were taken with the animal being offset for the viewer).

Made using
FaceTracker library from Jason Saragih (web.mac.com/jsaragih/FaceTracker/FaceTracker.html),
ofxFaceTracker addon by Kyle McDonald (github.com/kylemcdonald/ofxFaceTracker)
openFrameworks (openframeworks.cc/)
Unity3d (unity3d.com/)
Blender3d (blender.org/)

Background

Mirror neurons and on-verbal communication

Recently a neural mechanism has been discovered that explains how we get experiential insight of other minds.  Mirror neurons activate when we perform an action as well as when we watch someone perform an action, and thus have been implicated in imitation learning.  In addition to acting on the motor centers, mirror neurons have been theorized to play a role in ‘theory of mind’ concepts such as emotional recognition or contagion.  Emotional contagion is based on interpreting the emotional state of another being expressed through physical features.  Emotions (such as fear or sadness) have typical facial characteristics, and the mirror neurons theory would consider the neurons as ‘mapping’ the facial features of another being onto the respective areas in our own brain.  Evolutionary theory tells us that such direct mapping would be beneficial as it could help us perceive danger or threat more directly and quickly.  More complex evolutionary theories link mirror neurons to the appearance of cooperative behavior or emotions such as empathy and compassion.

The yet unanswered question is to what extent do mirror neurons might function between humans and animals, and do they function for more complex behaviors and emotions.  We do know that there is reciprocal behavior between different animal species, and a certain amount of mind modeling.  Studies of the predator-pray relationships tell us that survival in those roles depends on successful inserting of oneself into the mind of the hunter or the hunted, even to the point of mimicking its behavior.  “It was our ancestors’ skill at not only analyzing and imitating the nature of a given animal, but identifying with it, that enabled them to flourish in dangerous environments, both physically and psychically.” writes John Vaillant in The Tiger.  The ability to understand and communicate with other species is also apparent in our relationships with domesticated animals that rely on cooperation and emotional bonding.  Communication with animals in particular relies on a non-linguistic, body-based, instinctive and emotional aspect of expression. Social psychology studies have demonstrated that imitation and mimicry are pervasive, automatic, and facilitate empathy. The neural mirroring suggests that those mimicry exchanges are a bridge to inhabiting the other’s mind — and to the in-depth understanding of each other.

Mask: the provisional face

Throughout the world animal masks are used for their expressive and symbolic power – both ritually and in various theatre traditions. Often the mask is regarded as an instrument of revelation, giving form to the invisible. Many African tribes believe that the animal masks can help them communicate with the spirits who live in forests or open savannas. The neural mirroring mechanism suggests that these rituals might be far more than simply a symbolic performance — they are actually an embodied simulation of other creatures.

Symbolism is encoded into the designs of the masks to inform the spectators about the role and its dominant characteristics. These designs often exaggerate facial expressions.  For example The Senoufo people of the Ivory Coast represent tranquility by making masks with eyes half-shut and lines drawn near the mouth, and the Grebo of the Ivory Coast carve masks with round eyes to represent alertness and anger.

Nicola Savarese in “A Dictionary of Theatre Anthropology: The Secret Art of The Performer” discusses the ‘facial gestures’ expressive of emotional states that we share with animals. “The expression ‘to show one’s teeth’ is so rich in meaning that it has passed from physiology to proverb”, he writes and notes how we often intentionally ‘perform’ these facial gestures in order to communicate and thus turn our face into a mask: “The mask becomes a face and the face a mask. It is not the psychology of feelings but anatomy of forms which is being dealt with here.”

Mirrors, perception and the amalgamation of the physical and virtual

Part of my interest in this project is the combination of the virtual and physical world — inserting a layer of imagination into a world we know. The chain of causes and effects remains in place, although slightly augmented. The familiar is transformed into the uncanny, prompting us to see the mechanics of perception, interaction, and relationships with others anew.
“In a sense, mirrors are the best ‘virtual reality’ system that we can build,” said Marco Bertamini of the University of Liverpool. “The object ‘inside’ the mirror is virtual, but as far as our eyes are concerned it exists as much as any other object.” Dr. Bertamini and his colleagues have also studied what people believe about the nature of mirrors and mirror images, and have found nearly everybody to be shockingly off the mark.  (For example, no matter how far we stand from the mirror, our face in the mirror is always exactly half the size of our face: the mirror is always halfway between our physical selves and our projected selves in the virtual world inside the mirror, and so the captured image in the mirror is half our true size.)

To scientists, the simultaneous simplicity and complexity of mirrors make them powerful tools for exploring questions about perception and cognition in humans and other neuronally gifted species, and how the brain interprets and acts upon the great tides of sensory information from the external world. They are using mirrors to study how the brain decides what is self and what is other, how it judges distances and trajectories of objects, and how it reconstructs the richly three-dimensional quality of the outside world from what is essentially a two-dimensional snapshot taken by the retina’s flat sheet of receptor cells.
– from New York Times

Title

Konstanty Ciołkowski, a Soviet Russian rocket scientist, pioneer of cosmonautics, writer and philosopher used the phrase “all the universe is full of the lives of perfect creatures” in his “The Cosmic Philosophy” whose main idea is arriving at the absence of suffering for all the living beings in the Cosmos, for all the Universe.

Surveyor Suite is a tool for exploration and re-imagining of the urban environment. The devices are used for performative research and public events. The resulting video, writing and images are collected online.

Surveyor is an iPhone app for use with a portable mini projector. A projected man surveys his surroundings and catalogues everything around him. He has binoculars and a log book, in which he intermittently writes his notes. The user of the app can direct the gaze of the man, examining the particular site he’s in. The virtual gaze of the surveyor directs the attention of the people around him, be it to other people or to their surroundings. Playful tool of engagement at a distance, it creates a connection between the app user and the people viewing the projection, as they become implicated in the piece. In this case the user is the puppeteer, utilizing the classic performance technique where the eyes of the performer move to where he wants to direct his, and therefore the spectators’ attention. The gaze functions like a telephoto lens, zooming in on a camera shot.

The projection is the entry point for re-interpreting and re-envisioning the reality of urban environment. Used in a public space, the Surveyor becomes a public engagement tool that inspires personal involvement and facilitates public discussion. Activated by the imagination of the viewers, it is a catalyst that lets us see our surroundings anew, wonder what’s behind the walls and reevaluate things by inviting speculation and questioning through storytelling and imaginative thinking.

Work in progress:

Surveyor Suite is a part of Amateur Human project

Made using
Unity3d (unity3d.com/)
Blender3d (blender.org/)

Background

Lawrence Weschler talks about the ideas of Nicholas of Cusa (an early Renaissance mathematician and mystic):

To gain knowledge of God, one should just catalog everything — books, rocks, flowers, human emotions — so that by the end you have cataloged all of God’s creations. So Cusa, who is a mathematician, says: “Well, I suppose that’s a bit like an n-sided polygon inside of a circle.” In other words, you take a triangle inside of a circle, and you keep adding sides to it, and the more sides you add the closer it comes to being a circle. And yet, at the same time, you keep getting further away, because a circle only has one “side,” one line, and here you’ve got a million lines and angles. Cusa was the one to come up with this concept that at a certain point, you have to make a leap of faith–from the n-sided polygon to the circle, say. And that leap of faith is accomplished through grace, which is to say, for free.

This is similar to the theoretical physics’ quest for the “Theory of Everything”. As Margaret Wertheim points out, science is founded on the belief that things are comprehensible and that by the ingenuity of our minds and the probing of ever more subtle instruments we will ultimately come to know It All. But is the All inherently knowable?

In the quantum teleportation experiments physicists ‘teleport’ a photon — or replicate all the physical properties of a photon in a photon in another location. Borges writes about the map that is so detailed that is the exact size of the territory. In his story,the cartographers realize how pointless it became and the effort is abandoned, reminding us of the function of abstraction.

Shadowplay is a mobile game for the use with a portable projector. In this game, an animated image of a wolf is projected from the device. The user, using multitouch gestures, can direct the animal’s movement to interact with the environment it’s projected onto. For the ISEA exhibition a virtual wolf habitat is mapped onto the area of Albuquerque. The devices are distributed to participants, who are encouraged to explore this habitat, ‘puppeteering’ the wolf in the public. The game design will be shaped by the research done during the residency using the NMWA wolf data collection. Resources used by wolves as well as things they need to avoid are scattered around the ‘habitat’ area using geolocation markers. During the workshop, the participants learn about the wolves lives, environment and the history of their relationship with people.

A short video showing work in progress animation and touch gesture control (the password is ‘shadowplay’):

Animal myths and folktales have regulated human behavior towards animals for centuries. The myths formalize and generalize the culture’s relationship to them. They often help us to respect and understand the animal’s habits as well as encouraging us to keep our distance. The wolf — real and imagined — in particular has held deep meaning for humans. Revered and vilified, today it is often the victim of our collective imagination. The old vestiges of myths can lead to misunderstanding and mistreatment of wildlife: our stories and attitudes are in need of updating to reflect wolf’s and our place in the changing ecology.
Shadowplay aims to inspire a new perspective by giving the participants a way to not only familiarize themselves with facts about wolfs, but more importantly identify with the wolf, create an empathetic emotional connection to it.

The participants create new kind of ‘folktales’ based on what they learned and what they imagine, of how the wolves interact with their natural environment, with the human-built environment, with other wild and domesticated species and with people. They are specifically encouraged to pick apart the assumptions that the old myths left in popular culture and to see the animals as neither demonized or glorified. All these stories, writings and audio recordings as well as images and videos from the city explorations, are collected online and shown in the exhibition, contributing to collective new wolf ‘mythology’.

Play is the domain of innovation as well as imagination. As they explore with their projections, the participants are encouraged to point out the problems and propose solutions to managing meeting ours and wolves’ needs.

As they project on the city walls the participants become performers, drawing attention and potentially participation of others, turning their play into a public address. The sight of a wild animal is a rare and wondrous experience. It inspired the ancient humans to cover their walls with drawings of wild animals, and no less impresses us today. A projection of an animated animal in an urban setting carries on a part of that wonder and inspiration, and the reminder that these animals might become mere shadows in our memory if we do not protect them.

Capacity (Light Blue)
-horizontal raised surface a foot from the floor 7′ x 3.9′
-projection down from the ceiling
-16×9 resolution projector ( 2500 lumen or higher, preferably DLP and good color resolution)
-Mac Pro tower
-small prints on the walls

Pornographic Pursuit (Red)
-front projection on the wall
-projector mounted on the ceiling (4:3 resolution, 2500 lumen or higher, preferably DLP)
-Mac Mini
-sensor matte 3 x 4 ft on the floor in front of the projection, connected to the computer with a cable. The matte should be around 6 ft away from the wall. Sensor supplied by Karolina.

Stability (Orange)
-front projection on the wall
-projector mounted on the ceiling (4:3 resolution, 2500 lumen or higher, preferably DLP)
-waist-high pedestal in front
-sensor cube (supplied by Karolina)
-Mac Mini
-maintenance: requires daily changing and overnight recharging 4 AA batteries in the sensor.

All the Universe is Full of the Lives of Perfect Creatures (Dark Blue)
-vertically mounted lcd monitor (approximately 23″ diagonal)
-wall mount for the monitor
-Mac mini
-Logitech C910 webcam (or similar wide angle) (supplied by Karolina)
-half-silvered mirror and a box housing it and covering everything else (size depends on the size of the monitor)

Surveyor (Green)
-Optoma PK301 Pico projetor
-mounting arm
(such as this: http://www.amazon.com/Manfrotto-244-Variable-Friction-Bracket/dp/B00193W3Z2/ref=sr_1_2?ie=UTF8&qid=1322683369&sr=8-2
http://www.amazon.com/Manfrotto-035RL-Super-Clamp-Standard/dp/B0018LQVIA/ref=pd_bxgy_p_img_b)
-lcd monitor and a dvd player or computer playing back a documentation video from Surveyor’s use in the city

Pedestal (Light Green)
-Optoma PK301 Pico projetor
-mounting arm
(such as this: http://www.amazon.com/Manfrotto-244-Variable-Friction-Bracket/dp/B00193W3Z2/ref=sr_1_2?ie=UTF8&qid=1322683369&sr=8-2
http://www.amazon.com/Manfrotto-035RL-Super-Clamp-Standard/dp/B0018LQVIA/ref=pd_bxgy_p_img_b)
-shelf
-stool

+ Storefront Projection
Sniff or It’s You

-rear projection (about 3500 lumen or more, DLP projector)
-rear projection screen or other projection surface (window glass can be frosted for example)
-Mac mini
-speakers and audio interface to computer
-Kinect camera, mount and usb 2.0 extension

Amateur Human is a multidisciplinary exploration of the issues of consumption, environmental ethics and social responsibility. This multi-part project encompasses designing accessories that personalize and make public our relationship with the environment, investigation and documentation of the process of bringing such devices to market, and distribution of the accessories to the users, documenting and exploring facets of the consumption-related issues.

This project is exploring design’s potential, at the heart of consumer culture, to be subverted for other ends, such as critical and artistic exploration. The accessories are not meant for mass-production, but are conceptual design proposals, prototypes that ask what if. Their form is slightly absurd and humorous, making them non-didactic and inviting wide-ranging interpretations. The objects often rely on the voyeuristic transgression of private space turned public.

Another aspect of the project is the exploration of the user-end experience of the accessory. Each user’s perspective throws a little light on the entangled issues of environmental stewardship, social responsibility, and our desires, concerns and responsibilities regarding sustainability issues. These perspectives are captured in a long-form documentary, and in the interviews presented online.

Amateur Human received an Emerging Fields grant from Creative Capital.
Project’s website

Woodstock Digital Media Festival June 22-23, 2012

The Festival is many things:
A showcase of the most innovative, accessible, and thought provoking digital media
An engaging interactive exhibition and exploration
A memorable event that captures the energy from different perspectives – artists
educators, non- profits, and business leaders
A unique experience in Vermont as the perfect backdrop to engage in digital media

Karolina Sobecka – Human Moves, Animal Visions, April 20 – May 3, 2012

As part of the Film Society’s ongoing KinoTek programming stream, artist Karolina Sobecka will install several dynamic works at SuperFrog Gallery, just upstairs from the Film Society Cinema. A leading figure in the production of interactive installations, Karolina Sobecka uses animation, interface design and humor to comment on the ways that we use our bodies and gestures to navigate and understand our overtly mediated environments.

April 20-May 3, 2012

Each KinoTek series program is accompanied by an original article commitioned by SFFS. Media theorist and critic Akira Lippit (Electric Animal) will disscuss Karolina Sobecka: Human Moves, Animal Visions in an article to be published on this page.

CAM Raleigh

Several of my installations are included in Born Digital exhibition at Contemporary Art Museum in Raleigh which opens January 28th, 2012, and is up until April 30th.

CAM Raleigh’s newest feature exhibition, Born Digital, invites visitors to physically explore digital culture and movement-driven artwork—to exercise their creativity and act on their curiosity. Opening January 28, 2012, the exhibition showcases the contemporary, visitor-dependent art of 12 national and international pioneers of digital and new media art.
Featured Designers and Artists
Born Digital contributors include: Advanced Media Lab, Jacob Ciocci, R. Luke DuBois, Channel TWo, Brent Green, Ajay Kurian, LoVid, Cole Pierce, Dennis Rosenfeld, Daniel Rozin, Scenocosme, and Karolina Sobecka.

Yale Art School

Frank Pichel and I will be doing a artist presentation at Yale Art School January 24, 2012.
The presentation is titled ‘Context’ and we’ll discuss how consideration of the audience impacts the work.

Nuit Blanche New York: Bring to Light

I will participate in this year’s Nuit Blanche New York event: Bring to Light October 1st, 2011, 6pm to midnight. My two installation will be at storefronts at the opposites sides of Franklin Ave at Greenpoint Ave.

Bring to Light is a free nighttime public festival of art in New York City that takes place simultaneously with “nuit blanche” events in cities around the world. Inviting emerging and established artists to make site-specific installations of light, sound, performance and projection art, the event creates an immersive spectacle for thousands of visitors to re-imagine public space and civic life. Bring to Light will transform streets, parks and the industrial waterfront of Greenpoint, Brooklyn set against dramatic views of the Manhattan skyline.

Special Projects Award 2011

I received the 2011 Special Project Award from Princess Grace Foundation for the further development of Amateur Human.

NYFA Fellowship 2011

I’m happy to announce that I received this year’s New York Foundation for the Arts Fellowship in Digital/Electronic Arts! This year the Fellowships were given in the following categories: Crafts/Sculpture, Digital/Electronic Arts, Nonfiction Literature, Poetry, Printmaking/Drawing/Book Arts. For the list of all the fellows in all the categories, the finalists and panelists, see NYFA’s website.

Media Archeology 2011

“Sniff” was invited to the Aurora Picture Show, which this year presents the 8th annual Media Archeology Festival Thursday, September 15 through Saturday, September 17, 2011 in Houston. Taking place at venues across the city, including The Orange Show and the Museum District, the festival is presented in collaboration with the University of Houston Cynthia Woods Mitchell Center for the Arts. This year, the festival explores the ideas of games and play from the past and present.

Interaccion I/O/I

“Sniff” is up in Barcelona until the end of August 2011, part of the Interaccion I/O/I, an exhibition at the DHUB Museum. A little bit about the exhibition:

I/O/I. The senses of machines (Interaction Laboratory) proposes an experimental and educational look into the relationship between man and machine at a time when new generation interfaces include natural interactive features and social behaviors. It is an exploration of new areas where interaction design is developed and generates new needs, disciplines and experiences.

Amateur Human is a multidisciplinary exploration of the issues of consumption, environmental ethics and social responsibility. This multi-part project encompasses designing accessories that personalize and make public our relationship with the environment, investigation and documentation of the process of bringing such devices to market, and distribution of the accessories to the users, documenting and exploring facets of the consumption-related issues.

This project is exploring design’s potential, at the heart of consumer culture, to be subverted for other ends, such as critical and artistic exploration. The accessories are not meant for mass-production, but are conceptual design proposals, prototypes that ask what if. Their form is slightly absurd and humorous, making them non-didactic and inviting wide-ranging interpretations.

Another aspect of the project is the exploration of the user-end experience of the accessory. Each user’s perspective throws a little light on the entangled issues of environmental stewardship, social responsibility, and our desires, concerns and responsibilities regarding sustainability issues. These perspectives are captured in a long-form documentary, and in the interviews presented online.

Amateur Human received an Emerging Fields grant from Creative Capital.
Project’s website

Puff is the first of the accessories, and 3 more are in the works.

Human agency is the capacity for human beings to make choices and to impose those choices on the world. It is normally contrasted to natural forces, which are causes involving only unthinking deterministic processes.
The capacity of a human to act as an agent invests a moral component into a given situation. Human agency entitles the observer to ask should this have occurred? in a way that would be nonsensical in circumstances lacking human decisions-makers. (source: Wikipedia)

Computers afford us the ability to run processes and execute rule-based symbolic manipulation. In computer ‘games’ or ‘simulations’ we can represent how real and imagined systems work.

Interactive installations too are logical systems which create relationships between a few interdependent elements, one of which is often the viewer.

The workings of these rule-based representations can be partly exposed in a visual ‘debug’ mode. While the game scene shows the seamlessly integrated effects of the rules implemented in the system, the visual debug mode is designed to shows the effects of each of the rules, to literally ‘picture it’ and see it work. In this view they can be easily scrutinized to improve, detect anomalies and assess impact, but more interestingly they also form a raw and unmediated snapshot of the ‘conflicts’ or relationships from which the project is built.

While the outward expression is communicated to the audience as a narrative metaphor, the internal symbolism of the debug lines can be read on a different level. Even without the knowledge of what the debug symbols signify, they form an impression of bearing a meaning encoded in a visual language, behind which are rules of logic. The abstract debug symbols, interposed on the renderings of elements whose behavior is driven by the debugged rules, imparts to those elements the weight of their rule-bound necessity. At their meeting, the two pictorial spaces inform and enrich one another, the illusion is both exposed and justified.

Even though these simulated systems tend towards clean abstractions and explicit rules, they gain ‘noise’ as they are implemented and expressed in visual terms. The appearance of the debug symbols is subservient to their function, but it is arbitrary as far as it serves the preferences of perceptual analysis (and as far as being within the vocabulary of the software tool). Their color is meant to stand out from the other colors, their size is meant to be seen from a perspective at which the given phenomenon can be observed, their form is chosen to communicate the position, direction, movement or other property of the element. Thus their arrangement and the resulting composition are the pictorial translation of the stakes portrayed in the work.

Alex Galloway writes about games being ‘the occult logic of software’.

Code hides itself in the very act of consummating its own expression. A game expends itself in the very act of its being played. And so the game retreats from its own essence. But only in such a way as to be more true than the essence could ever be.

Eddo Stern writes about the ‘artifacts’:

Most fantasy game designers would regard visible signs of any technological underpinnings as unwanted anachronisms that would threaten the constitution of the immersive fantasy they are attempting to construct. The resulting by-products of this problem can be found in the designers’ introduction of metaphors that function to assimilate unwanted technological residues into the narrative diegesis.

•A note on “Artifacts”
I am borrowing the term artifact from computer science where the term is used in reference to undesired cosmetic disturbances such as jagged edges or dirty patches in an image file (common in compressed digital video or jpeg images for example), excess noise or hiss in a sound stream, or unpredictable ASCII characters in a text file. Artifacts differ from bugs, which are usually caused by programming mistakes; artifacts don’t prevent functionality per se, but cause an unperfected aesthetic disturbance.

The images in this project show the visual ‘debug’ mode of the project Forth in which deterministic forces are pitched against the questionably ‘willful’ efforts of the people.

Forth is a public art installation commissioned by the University of Central Florida for the Institute for Simulation and Training.

Forth employs computer simulation techniques, interactivity and real time audio synthesis to generate a dynamically changing ocean scene. The seascape projected on a large curved screen is subject to the local weather conditions: the wave height and the atmospheric conditions reflect the weather sampled in Orlando and at the Florida coast. Small row boats containing groups of people advance across the expanse of open water, and narratives emerge within each vessel as the rowers try to stay afloat and on course in turbulent waters. A second screen in the lobby displays an underwater view of the scene, offering an unexpected perspective. Immersive sound heightens the richness of the experience, responding to the weather, and to the flow of visitors through the lobby.
The viewers in the lobby are tracked by a camera and are represented in the projection as paper boats floating on the ocean. The rowers may pick up these paper boats — bridging again the real and virtual worlds.

Software development for Forth was led by James George.  Sound design was created by Michael McCrea.

Project’s site

A 13-min video documenting the installation:

And a few captured screenshot movies of the projection content:

The movie above shows a few of the ‘features’ of the project:

• the weather changes according to the weather conditions in Orlando
• the waive-height of the ocean changes according to the weather conditions at the Florida coast and in Orlando
• color palettes change every week, with a small variation every day, transitioning between different color treatments for daytime and nighttime.
• the direction of the boats changes slowly over the course of the year
• the distribution of the boats varies all the time and is designed to create interesting compositions
• people in boats lean to stay upright, compensating for the tilt of the boat
• people’s rowing is also adjusted to the weather conditions
• people walking in front of the projection in the lobby are represented as paper boats floating on the ocean in front of the virtual camera
• the paper boats left on the ocean by the people present in the lobby can be picked up by the people in boats
• two of the ‘directorial gestures’ are shown in the demo: the ‘no rowing gesture’: people in the boats stop rowing and look at the viewer and the ‘stray boat gesture’: one of the boats turns and starts going in a different direction than all the other boats.
• the underwater scene is shown on an LCD monitor opposite of the projection

Puff, is a personal emissions visualizer: a lamp in the shape of a cartoon cloud that attaches near the exhaust pipe of one’s car, and dynamically changes color indicating the amount of pollution the car is producing. An accompanying iPhone app provides visual feedback for the driver, and keeps track of the emission data over time.

Puff is a part of Amateur Human project.

Amateur Human is a multidisciplinary exploration of the issues of consumption, environmental ethics and social responsibility. This 3-part project encompasses designing personal devices that visualize and make public our impact on the environment, investigation and documentation of the process of bringing such devices to market, and distribution of the device to the users, documenting and exploring facets of the consumption-related issues.

This project is exploring design’s potential, at the heart of consumer culture, to be subverted for other ends, such as critical and artistic exploration. The devices are not meant for mass-production, but are conceptual design proposals, prototypes that ask what if. Puff’s form is absurd and humorous, making the device non-didactic and inviting interpretations.

Decisions made at the design stage in industrial production, shape the world around us: materials, fabrication methods, live cycle and environmental impact are considered. Amateur Human , through documentation of various aspects of production, attempts to elucidate the production processes behind the objects around us, promoting literacy of how things are made and consideration of how things could change.

The third aspect of the project is the exploration of the user-end experience of the device. Each user’s perspective throws a little light on the entangled issues of environmental stewardship, social responsibility, and our desires, concerns and responsibilities regarding sustainability issues. These perspectives are captured in a long-form documentary, and in the interviews presented online.

Amateur Human received an Emerging Fields grant from Creative Capital.
Project’s website

Created in Banff National Park, part tourist-snapshot, part picturesque-postcard-vista, the 10 postcards in the series are a reflection on being out of place and on the resulting perspective which affords one to see the society-encoded social rules in a new light. Short annotation on the back of the postcards are Alice-In-Wonderland-esque descriptions of a state of mind rather than a place, and tell a sequential story.

Chris Rojek in “Indexing, Draggin and the Social Construction of Tourist” notes:

A tourist sight may be defined as a spatial location which is distinguished from everyday life by virtue of its natural, historical or cultural extraordinariness. Tourist sights are predicated in a binary opposition between the ordinary/everyday and the extraordinary. This opposition is culturally constructed.
(…) Without doubt the social construction of sights always, to some degree, involves the mobilisation of myth. As a social category ‘the extraordinary place’ spontaneously invites speculation, reverie, mind-voyaging and a variety of other acts of imagination. Sights have produced a discursive level of densely embroidered false impressions, exaggerated claims and tall stories. Of course, it is enormously difficult to disentangle this tradition of deliberate fabrications from our ordinary perceptions of sights.

The remoteness of the ‘extraordinary place’ requires abandoning our everyday life routines and replace the normally practiced rules with ones imposed by the place.

Sniff is an interactive projection in a storefront window. As the viewer walks by the projection, his movements and gestures are tracked by a computer vision system. A CG dog follows the viewer, dynamically responds to his gestures and changes his behavior based on the state of engagement with the viewer.

Sniff, simulating the visceral satisfaction of reality’s responsiveness and dynamism, is also an exploration of engagement of two different planes of understanding, and of relationships created by body’s presence in an environment. Dog’s behavior externalizes the process of assessment, evaluation and testing we perform every time anything new enters in the scope of our experience. Sniff has us unwittingly enter into an exchange simply by following the basic instinct of stopping and looking at something that is paying attention to us. A tension is produced by a mixture of expectations, reaction and re-assessment.

Project’s site

Software development for Sniff was led James George.

Sniff production was supported by a Finishing Funds grant from the Experimental Television Center. The Experimental Television Center’s Finishing Funds program is supported by the Electronic Media and Film Program at the New York State Council on the Arts.

Created at the Interactivos?08 at Medialab-Prado
This system is a prototype for a portable image recording system based on multiple cameras positioned along a path. The cameras will be controlled by a microcontroller, which can assign a time delay to each camera independently. The rig is modular and reconfigurable allowing for arranging it in any shapes. Based on cheap digital disposable cameras, this aparatus will lend itself to all kinds of temporal-spatial experimentation.

More about the ideas behind the project

In collaborataion with:
Sofy Yuditskaya, Emanuel Andel, Gwenn Joyaux, Tais Biels Rey, Laura Gabriela Olalde Verdes and with help from many others.

In this first demonstration we explore it to re-conceive or visualize a spatial perception which expands the body’s point of view in space.
We recorded multiple sequences with cameras arranged on quarter of a circle arch. For the interactive display we projected an image on the wall. As a person walks by her position is tracked and the picture displayed changes perspective (scrolling through our sequence) corresponding to the angle at which the person is viewing the picture. In effect the viewer can see a ‘moment’ from many points of view, physically moving around it to explore it.
Once the viewer scrolled through the entire ‘moment’, the sequence of another moment will be loaded, so that the viewer can walk through the story moment by moment.

A basic illustration of the interaction in the installation:
as a person walks her position is tracked and the picture changes perspective scrolling through our sequence. Once the viewer scrolled through the entire ‘moment’, the sequence of the consecutive moment is loaded.

The movies below show the series of ‘moments’ that the viewer would ‘walk through’ to see the story.
We shot them at Mataderos, an old slaughterhouse now being converted to a cultural center. We did our best to align the cameras so they’re all centered on exactly the same point and at the same angle, but with no good visual reference the cameras were pointing at slightly differnt angles and the pictures had to be re-aligned in post. Even though I’d rather it wasn’t a result of the lack of control, I like the resulting visibility of that imperfection.

The narratives we shot were very abbreviated, since we didn’t have time for any real production. In this sequence the tourists are getting their pocket cameras stolen as they point their SLRs at some attraction. The story is unfolding in both time (in consecutive moments) and space (as some information is only visible from one angle not another).

Emanuel documented the process at tagr.tv and flicker set, there are also lots of pics at Interactivos? flickr stream.