Introduction

Projected human figures walk along the window paralleling the movement of the pedestrians. Ocasionally one of the figures stops and points at a pedestrian passing in front of the window. This sudden gesture of recognition, confrontation or accusation directed at him startles and singles out the viewer. The other projected figures stop and stare at him, silently questioning, scrutinizing, judging.
When the singled-out viewer moves in front of the display, the pointing figure follows him with the extended arm and finger, and as the other projected figures, continues staring.
When the singled-out viewer leaves the display or stays there long enough, the pointing figure lowers his arm, composes himself and continues on his way along the display. Other figures also resume their walk..

Background

“Storefront stories” is a series of interactive projections in storefront windows. The installations explore the process of encounter and behaviours in public space. Creating situations inherently full of tension and ambivalence, they focus on the social mechanism and the line between the real and constructed social actions and expressions.

The animated imagery is rear-projected onto a frosted full-length window facing a public walkway. The projections are of simply rendered CG characters whose behavior is scripted in a game engine to respond dynamically to the presence and behavior of the viewers (which are analyzed witha computer vision software).

In some scenes the interaction is very direct, in others it’s more subtle, and almost unnoticeable. The characters are life-sized: the space created by the installation becomes a blend of the real and virtual, triggering all the automatic reactions appropriate in a public space, while remaining a metaphor that relies on the suspension of disbelief, leading the viewer to examine their relationship to their social and physical environment.

How it works

IT'S YOU is composed of two main components, a video tracking system and a game engine for real time graphics. The video tracking system is built in openFrameworks (which is a free and open source C++ library); the game engine is Unity3d (also free).
The low-poly people are modeled and animated in Blender3d (free and open source). The modesl with attached animated actions are imported into Unity3d, without smoothing to retain its polygonal appearance.

People on the sidewalk are monitored by an IR camera in openFrameworks. In oF each individual person is isolated and assigned a unique id for the duration of their interaction. Each persons’ position information is continually sent to Unity3d via OSC networking protocol. In Unity, the behavior of the animated characters is modeled to pick a person who will be singled out randomly and for the figure pointing out the person. All the other figures assume the group behavior around the pointed out person.