On a late summer night in March, a handful of people from San Francisco’s Obscura Digital stood at a cruise ship terminal a half mile from the Sydney Opera House in Australia. Armed with powerful projectors, they took aim at the building’s iconic sails.
Inside the concert hall, other team members trained cameras on conductor Michael Tilson Thomas and the YouTube Symphony Orchestra. They flashed these images back to the terminal, where a computer program distorted them to match the geometry of the sails. Instantly, the conductor and his orchestra appeared on the building in perfect time and perspective.
Jawdropping, sure, but also becoming more and more common. Projection mapping has taken over European cities in the last year and made dozens of appearances elsewhere. And it's no surprise why. The effect is bright, three-dimensional and transforms buildings in show-stopping ways.
“Most people think [about projectors] in terms of putting a rectangle on the wall,” says Travis Thelkel, chief creative officer of Obscura. “[Projection mappers] see them almost as a blanketing tool. We think in terms of using 10 or 20 or 30 projectors to canvas an entire area and do whatever we want with it. It's a transformation tool rather than a display system.”
Seeper’s projections have taken over public spaces for an AC/DC-inspired Iron Man promotion (left) and Xbox 360 (right).
To understand how it works, we talked with three leading firms in the field: Obscura, Klip Collective from Philadelphia and Seeper from London. (Their names and work appear in this article in no order of importance.) They revealed a process that’s intensely mathematical, yet almost equally playful.
Projection mapping 101 starts with your parents’ slide projector. Those hot, cranky machines beam light through a slide onto a flat screen. If your projector is level and roughly perpendicular to the floor, you see a fair replica of the image.
Take a similar image and project it on the Sydney Opera House, and it ends up stretched. Michael Tilson Thomas now looks like he is standing in a funhouse mirror, and the rest of the orchestra is a sea of swirls. A flat, rectangular building doesn’t solve the problem either. Any surface feature, like a balcony or window, still plays havoc with the image. You also have to account for the odd size of the canvas and the angle of projection.
To get around this, you “map” the object, or create an accurate 3-D model of its geometry and surface features. This can be easy or difficult. Because the Sydney Opera House was constructed before the arrival of CAD, Obscura had to find 3-D data from a helicopter scanning of the site. On the other hand, the earliest video projection seems to have been created in 2003 by Ricardo Rivera of Klip, who used a single projector to map his kitchen. His firm has gone on to patent the process and project onto some of the odder objects in the field.
Once you have your map, you divide your projection canvas into “cards,” based on the polygons in your model—much like a special effects artist might composite a scene. Then you distort your video to match that geometry.
Finally, it’s time for the fun stuff. In addition to projecting on virtually any object, you can also transform it. Projection mappers love to play with trompe-l’oeil effects straight out of the Renaissance. They use shadows and varying levels of brightness to cause buildings and objects to come alive. Three-dimensional rabbits jump from ledge to ledge. People open windows and peer out. A whole façade collapses and then rebuilds itself brick by brick. If you’ve never seen anything like this before, go straight to YouTube and find Seeper's “Battle of Branchage.” It’ll amaze you.