“The system can accurately measure your skeleton within a few millimeters,” he says.
Markerless motion capture by Organic Motion at the SONY Wonder Technology Lab (SWTL) in Manhattan. Image courtesy of SWTL.
Organic Motion may not be one of the better-known players for 3-D gestural interfaces, but its“markerless motion capture” system is easily one of the more elaborate and precise. It can track every movement of a person—down to the fingers. Clients have used it to create everything from military simulators to interactive dance exhibitions.
If you ask Tschesnok, and many other technology makers, 3-D gestural interfaces are the future. More than eight million of us can already move around our living rooms and play games with Microsoft Kinect. Within a year, we’ll be able to buy gesture-controlled computers, dockable 3-D interfaces for devices like the iPad, and maybe even TVs equipped with 3-D cameras.
“I think in five years time, when you buy a computer, you’ll get this [3-D gestural interfaces],” John Underkoffler, the chief scientist of Oblong Industries, Inc. recently told a TED audience.
As futuristic as it all seems, technology like this has been around much longer than you might think. A successful experiment at the MIT Media Lab called “Put That There” appeared as early as 1980. It involved pointing at a map and requesting that ships be placed in a variety of locations. It was followed by a large number of academic systems and experiments that used everything from infrared and stereoscopic cameras to eye-tracking devices and glove-reading systems.
Oddly enough, it took a Hollywood movie to generate popular interest in the field. In 2002, Alex McDowell, the production designer on Minority Report, hired Underkoffler, then at MIT, to abandon thinking about real-world limitations, and instead describe a gestural interface from the year 2054. In a sense, it was an R&D project that sold the public (or at least a substantial proportion of its geeks) on the concept.
“When that movie came out, every client wanted the interface in Minority Report,” says Dave Small, principal of Small Design Firm, which specializes in digital displays.
Underkoffler only made one mistake. The technology would arrive much faster than 2054. At the high end, you have companies like Organic Motion, whose product is not really a dedicated interface system, though it does deliver motion-capture data that can be used for that purpose. At the extreme other end is 3Gear Systems, a MIT project that has prototyped a whimsically-colored set of Lycra gloves that work with any ordinary Webcam. Theoretically, the gloves would cost about $1 to produce.
In between these, the adventurous art director can choose from an array of moderately-priced systems. Some, like the popular ones from GestureTek, work more or less out of the box. But most consist of middleware, products that don’t do anything themselves, but instead deliver data that designers and software developers must reinterpret to use.
Kinect is one example. It offers a platform that game developers can use to create experiences based on speech and gesture. Big rivals of it in the consumer space are likely to be Softkinetic and Optrima, two companies that offer an inexpensive “time of flight” technology that determines a person’s position by measuring the speed at which photons bounce off him or her. The companies plan to place it in Internet connected TVs, making them, in effect, gaming platforms.
OK, so there’s going to be a lot of this stuff, but will it see widespread use?
The answer in some areas is already yes. Gamers love the technology. “If you’re engaged in a game, something playful, or a virtual world, moving physically through can be a thrilling experience that can make a huge difference,” says Tom Hennes, principal of Thinc Design.
On the more public side, museums and educational institutions have quickly latched onto 3-D gestures. Physical movement has been shown to enhance memory, which has created a lot of commissions for interactive learning systems–though the interfaces tend to be very simple. Also popular are 3-D interactive display ads. Last year saw a number of projects in this area, including one from Disney that promoted The Sorcerer’s Apprentice movie with a game that had users throwing fireballs around a castle.
“One of the main applications so far has been interactive marketing,” says Michel Tombroff, CEO of Softkinetic, whose software helped power the Disney display. “You let people play with the system for a few minutes, and it transforms passive interaction into active. Advertisers like a captive audience for a few seconds, and once you play something like that you never forget the brand. And you get a crowd.”
Still, there is cause for skepticism. Success in games, museums and display advertising can be deceiving—and even a red flag. Those environments tend to produce interface concepts that don't translate well to the consumer market. Gamers have long adored 3-D worlds, but online shoppers have never agreed. You could say the same for interactive tables (a staple at museums), back-projected glass touchscreens (once popular at tradeshows) and RFID (which was going to revolutionize the shopping experience, but didn't).
Even among those who have successfully deployed the technology, caution abounds. “I may eat my words, but I think the file structure in your computer has stabilized to the point where you can find things quickly,” says Hennes. “Adding a 3-D interface to that, unless you have massive data to sort through, is more of a distraction.”
A second factor may be more mundane but critical: comfort. Those who envision people waving their hands in front of themselves to navigate their TV, PC, or refrigerator interfaces are likely mistaken. It’s long been known, since the age of the telegraph in fact, that human arms don’t like to be held up for any length of time making small, precise movements. The result is a painful condition known as “gorilla arm,” which destroyed a promising touch technology in the 1970s.
Still, many of the products coming out seem well aware of the problem, and the sheer volume of efforts is an encouraging sign for the technology. Some may fall squarely into the gorilla arm trap, but 3-D gestural interfaces are going to have a trial run on just about everything. You never know what will stick and turn out to be useful.
Small, for one, has been through his share of interface innovations and treats it all with a bit of sang-froid. “There are some things that gestures are great for,” he says. “Other things work really well with a button.”
In other words, 3-D gesture interfaces may not be the future, but they’ll be a noticeable part of it. Especially over the next few years. ca