Loading ...

No matter how you view it, and we’ll get to your options in a moment, “Pearl” is a charming animated short. It places you in the passenger seat of a car, as you watch a man and his daughter drive across the country, following their musical dreams. When seen on YouTube, it tells a straightforward story that’s both touching and satisfying.

But “Pearl,” which was directed by Patrick Osborne and produced by Google Spotlight Stories, is anything but straightforward. First, it was primarily made for virtual reality (VR), not video. And if you watch it multiple times in that format, you’ll notice that the events change depending on where you look. That’s because “Pearl” is an interactive story—one in which the user helps determine the flow and outcome of the narrative.

In the interactive story community, “Pearl” is not only one of the best-known productions, but also one of the most controversial. The problem is that you can experience it for the first time without realizing you’re making choices about how the story unfolds. Is a story truly interactive if the people controlling it don’t know they’re doing so?

Scenes from Google Spotlight Stories’ “Piggy” (left) and “Pearl” (right). Directed by Jan Pinkava and Mark Oftedal, “Piggy” features a pig that responds to the viewer’s gaze as it is being tempted by a cake. © 2018 Google Spotlight Stories. The Emmy-winning, Oscar-nominated “Pearl” was directed by Patrick Osborne. Set in a hatchback, it is about a father and a daughter who chase their dreams. At the time of its release, “Pearl” broke new ground for Spotlight Stories in both storytelling and technology, with the most shots, sets and characters, along with custom lighting, effects and interactive surround sound in every shot. The short is anchored by the original song “No Wrong Way Home,” written by Alexis Harte and JJ Wiesler. © 2016 Google Spotlight Stories.

The interactive story world bristles with such controversies, and always has. People have been arguing about the genre long before there was much to argue about. Some believe vaudeville marks its beginning. Others see it as a digital phenomenon that began with 1960s-era computer stories. Academic departments on “computational storytelling” were launched in the 1990s, dissecting everything from choose-your-own-adventure games like Zork to hypertext novels on CD-ROM. And the dividing line between games and stories is a source of endless wrangling.

Today, however, the debates are fading into the background, as a new wave of creators has taken the field. Their work has filled museums, invaded festivals, shown up on TV and even dropped onto our phones. VR experiences are drawing long lines at festivals and amusement parks. And successful interactive productions like Punchdrunk’s Sleep No More have even had lengthy theatrical runs.

Rather than trying to nail the genre down, then, it’s probably best to think of interactive stories as fitting into a continuum of narrative approaches, each as valid as the next. At one end, artists are trying to create traditional, movie-like experiences with strong characters and emotional effects. At the other, people are creating stories that are shaped by everything from individual user input to collective audience data. Some don’t even seem story-like at all.

Google Spotlight Stories is probably one of the best places to find the traditional approach. Under the guidance of Pixar veterans Jan Pinkava and Karen Dufilho, it is providing advice, financing and technology to creators from a wide range of disciplines, including live theater and traditional filmmaking. Together, they are trying to not only tell great stories, but also solve the technical and structural problems associated with audience attention and story flow.

Their recently released “Piggy,” for example, is an interactive comedy, in which a jogging pig passes a cake. Every time you look away from the pig, it tries to steal the cake. Catch it doing so, and it looks sheepishly at you.

“Timing is everything in comedy,” says Pinkava. “But here, the time frame is not fixed and depends on where the audience is looking.”

On the far other end of the spectrum, you can find popular VR experiences created for movies, like the stunning “Coco VR” from Pixar and Magnopus. It allows friends to explore the film’s mythical Land of the Dead together. They can walk around on foot, take a gondola ride over the city skyline and meet characters in locations from the film. “Coco VR” does not direct user attention anywhere. It doesn’t have a narrative arc, but still makes for a magical experience.

Is that a story? It depends on who you ask.

Up to four people can join together in “Coco VR” (left), by Pixar and Magnopus, to explore the world from the film Coco. Courtesy of Disney•Pixar. Webby Award–winning transmedia project The Trip, by creative duo The Kissinger Twins, is about a faked moon landing. Combining an online interactive film, a short cinema film, a concept album and a large-scale photographic series, The Trip also includes a storytelling experience on Instagram (right).

“It’s my opinion that storytelling in VR does not have to be about telling users where to look and where to go in what order,” says Kalan Ray, development supervisor at Magnopus. “You can use things that are uniquely available to VR instead to make storytelling better.”

In addition to different approaches, interactive stories come on a bewildering range of platforms. To help make sense of them, Kamal Sinclair, director of Sundance Institute’s New Frontier Lab Programs, recently made an extensive survey of technology-based art and storytelling through a commission from the Ford Foundation. She interviewed more than 100 practitioners and reviewed more than 1,000 articles on the topic. In the end, she identified at least 21 subgenres, ranging from the obvious, like VR and augmented reality (AR), to the most avant-garde, such as artificial intelligence (AI)–driven and even olfactory experimental narrative (yes, you read that correctly).

“A story doesn’t have to be linear or even have an author,” explains Sinclair. “But it should create a journey experience. Whether there is one possible outcome or 400, at the end, you should be taken through an arc of some kind.”

With all that in mind, it’s probably best to meet a few practitioners. If you’re new to the genre, a great place to start is the Kissinger Twins’ The Trip. Made by Kasia Kifert and Dawid Marcinkowski, it is an interactive fantasy about faking the moon landing that appeared on the web, Instagram and as a mobile app.

What made it successful? For one thing, the two sourced a huge amount of found footage and knitted it together with an appealing narrative. The subject matter, as faux documentary, is also well suited to an exploratory format. And it’s just plain interesting. Even if you’re not a conspiracy theorist, Kifert and Marcinkowski do a good job of presenting the main character and imagining what a faked moon landing would have involved.

Another successful independent voice is the United Kingdom’s Martin Percy. He has his own take on interactive storytelling, seeing it as a genre where certain types of stories—such as education and documentary—work very well, while others—like romantic comedy—do not. His 2017 educational experience Virry VR: Feel the Wild, for example, teaches children how to care for animals. He also directed Lifesaver, which stars Daisy Ridley of Star Wars fame in her first credited role. Made using Adobe Flash, the film provides a simulator in which you save the life of someone using CPR.

“Survival works really well in interactive because it’s not about character development; it’s about saving people,” he notes.

Marketers have also gotten into the act, not least because studies have shown that people are more likely to remember interactive stories than traditional ones. Using cloud-based platforms like Eko Studio, brands ranging from Coca-Cola to MTV have been creating choose-your-own-adventure content or stories that you can see from multiple perspectives.

As you might expect, brands are not, however, the ones pushing the envelope in interactive stories. If you want to find those who are, you need to look at projects sponsored by places like Google, YouTube, Sundance Institute’s New Frontier and the National Film Board of Canada (NFB).

The NFB, for example, produces anywhere from fifteen to twenty interactive stories every year. Many are what’s known as transmedia projects, existing in multiple formats, including physical installations and social media presences.

“We’re trying to bring a perspective on how the world is developing,” explains Hugues Sweeney, the executive producer of the NFB. “Many of our projects are about creating reach and discovery, but we also have a mandate to push the boundaries.”

One recent example is Beyond Ice, an interactive ice sculpture at the Canadian Museum of Nature, which invites visitors to touch a real iceberg—and interact with a story that contains film, animations and other elements highlighting the biodiversity of the Arctic. It will reach 500,000 people each year for the next ten years.

Lifesaver, an interactive film by Martin Percy, throws the viewer into a situation where someone will die from a cardiac arrest unless the viewer saves her or him. Shown here is a typical choice in Lifesaver (left), featuring the first professional performance by Daisy Ridley of Star Wars fame. Courtesy of Resuscitation Council (UK). Upon entering the Canadian Museum of Nature’s Canada Goose Arctic Gallery, visitors will be transported to the Arctic thanks to the interactive installation Beyond Ice (right). Created in collaboration with the National Film Board of Canada, Beyond Ice provides a sensory experience of the region’s sounds, light, sights and even touch—with slabs of real ice. Arctic scenes and animations by Inuit artists are projected on the ice, bringing the installation to life as visitors explore the ice up close.

On the other end is the NFB’s “10 Naked Truths,” a simple Facebook project that explains how false claims spread on social media. It features a video that seems entirely believable when it is watched silently (most videos on Facebook are experienced without sound). Then, when you watch it with sound, you learn the recipe for creating fake news.

In contrast to the NFB’s broadly popular approach, Sundance’s New Frontier leans hard into experimental narrative. Through fellowships and other support, the group works with a wide range of artists across platforms that include AI, data, Alexa and even biometrics. A current fellow, Stephanie Dinkins, for example, is iterating on a previous project, Conversations with BINA48, which aims to teach one of the world’s most advanced AI robots how to express what it means to be an African American woman in the United States.

And all of that simply scratches the surface. In bedrooms, garages and universities alike, makers and creators are hard at work telling stories across every conceivable platform.

However, not everyone is sold on the genre. Some point out the failure of earlier types of interactive stories. Others criticize them for lacking the strong emotional experiences provided by movies. And, for all the shouting, relatively few interactive stories have seen widespread commercial success.

“There’s an industry-wide conversation that some of the hardware for things like VR is still catching up to the creative vision,” says Richard Cardran, cochair of the Experience Design Working Group at the Academy of Television Arts and Sciences. “But I’m excited about things like ARKit 2 from Apple. Imagine an AR mystery using your phone’s screen to reveal clues such as a murder weapon on your table and a chalk outline of the body on your rug, both rendered in real-time 3-D. Very impressive.”

Cardran’s realism mixed with optimism is common in the field, where nearly everyone points out that new genres often take a while before they gain a wide audience. But whenever so much talent and resources come together for a common purpose, someone usually cracks the code that enables a genre to go mainstream. We don’t know if that will happen in this case, but in the meantime, there are many interesting interactive stories to see.

“A lot of people spend a lot of time pontificating about what should be done or what will happen 20 years from now,” says Ray. “I’m much more interested in people who are eager and optimistic about the awesome things we are doing today.” ca

The author of this article would also like to thank Jason Brush, Shawn Johnson, Lori Schwarz and Eric Shamlin—all of whom provided invaluable assistance in understanding this complicated topic.

Joe Shepter is a freelance writer specializing in travel and interactive media. He has worked with Adobe, Oracle, Whirlpool and Coca-Cola, among others.

With a free Commarts account, you can enjoy 50% more free content
Create an Account
Get a subscription and have unlimited access
Already a subscriber or have a Commarts account?
Sign In

Get a subscription and have unlimited access
Already a subscriber?
Sign In