When it comes to augmented reality (AR), Havas, TBWA, MRM and other ad agencies have set up “labs” where creatives and technologists experiment with people’s likenesses and surroundings using computer-generated images.

As much fun as that sounds, the work is challenging, and, as the term lab suggests, the possibilities of AR—especially web-enabled experiences, which require no app download—are just starting to be explored and understood for advertisers. But the industry has made significant inroads. In the past few years, brands and their agencies have begun using filters to help sell lipstick, sunglasses and even IKEA furniture, and recently, the likes of Instagram, Pinterest and YouTube have launched AR features that enable users to virtually “try on” products.

“Try-on technology is why AR is the biggest thing that has happened to the mobile industry,” says Dom Heinrich, senior vice president, global executive director of LAB13, MRM’s innovation studio, which has locations in Frankfurt, New York City, Detroit, London, Manchester and Tokyo.

“The best-case uses of AR solve a consumer problem, like [wanting to see] how a couch would look in your condo before buying it,” says Heinrich. “The challenge for creatives is you can’t just get a copywriter, an art director and a technologist in a room together and say, ‘OK, now apply AR.’ The technology is complicated and requires understanding how AI can be used with it too. You want AR to seem invisible to the experience, not be the experience.”

He points to the try-on tool from L’Oréal, a McCann client. Leveraging its 2018 purchase of ModiFace, a developer of 3-D makeup simulations, the beauty brand invites visitors to use live video or upload a photo of themselves to the L’Oréal website to see what some of the brand’s shades of eye shadow, lipstick, hair dye and more would look like on them. As a direct result of this feature, conversion rates have tripled, according to L’Oréal, which has also partnered with Facebook and Amazon to bring its try-on experience to users on these platforms.

“It is one of the great examples of what the tech can deliver. I hope it helps our industry see advertising in a different light—that it doesn’t have to be about sending a message, but rather, giving people a tool in their hands that benefits them,” says Heinrich. “And isn’t that what advertising should do? Creating brand awareness and loyalty is what advertising does best, and creating services for consumers to solve their problems is exactly delivering on that. Personally, I’m fascinated that advertising is still seen as websites, TV commercials, and digital and physical adverts, and technologies are seen as ‘innovative,’ while it is actually about creating value for consumers through experiences.”

But AR offers more than just utility or frivolous fun, like the many filters that imagine what you would look like old, without hair, merged with someone else’s face and so on. Newer effects—like Snapchat’s Ground Transformation, which changes the ground to water or lava—could be a building block to creating immersive, virtual reality–like worlds.

Also providing inspiration for more-immersive advertising in this format? AR influencers. They have gained profile through Facebook’s software-sharing Spark AR platform, on which creatives of all stripes can create AR effects.

Among them is Irish filmmaker David OReilly, who created the animation sequences for the fictional video game in the futuristic Spike Jonze–directed film Her. He is on the radar of many creatives for recent work like Simulation, an AR filter that superimposes your face onto a fetus and then a corpse.

“He builds complete worlds that are as close to VR that I’ve seen,” says Harry Bernstein, chief creative officer at Havas New York and its digital and social media agency Annex88, which he founded. “As much as we have this cool lab with very experienced creative technologists, there are random creatives and kids all around the world making really interesting lenses. It is really in the hands of the user, and I find that inspiring. We’ve even reached out to a couple of creators to see about collaboration on a few ideas.”

Creating more-immersive worlds is where he hopes AR will go in advertising, he says. However, Bernstein cautions that “you need visionary marketers who understand there has to be a balance between numbers—like how many people engaged with a filter, downloaded it and were converted by it—and building a brand.” He advises brands to ask themselves questions like, “Is your brand technologically part of someone’s journey?

Is what you’re doing with AR creating social currency with consumers? Are you building a brand that is going to be selling products five years, not just five months, from now?” He says that while the numbers are important, “interesting stories are what build brands and social currency with consumers.”

Take the way that adidas Running, as part of its partnership with the International Space Station U.S. National Laboratory, promoted testing of its next-generation cushioning technology in microgravity aboard the lab. An Annex88-created filter virtually launched users into space. First, in selfie mode, users had a helmet placed over their head. When they tapped on the helmet, it closed and began to shake, simulating launch. For the back-facing camera, the agency designed a special spacecraft that launched from inside a silo that users could move around their screen and place wherever they wanted.

Unveiled in November 2019, the filter was used nearly 9,000 times, which led to more than two million impressions—all from an organic Instagram story. But what Bernstein likes most about it is that “AR was used as a pure storytelling device, telling a clear and simple narrative in a fun and engaging way that put the user in the center of the action.”

Building on the Pokémon Go craze that had people of all ages on the hunt for imaginary critters in the real world, other brands are using AR to create immersive storytelling and branding through gamified contests.

Licensing the Monopoly trademark from Hasbro, online financial institution Ally Bank set up Monopoly games in Chicago, New York City, Seattle, Charlotte, Dallas and Detroit in 2019. On the streets of each city, players could find a real-life statue of an oversized token in Ally’s vibrant shade of purple, like top hat, Scottie dog and car, and large tiles affixed to the ground that resembled the board game squares GO, Free Parking, Community Chest and more.

By scanning the squares with their phones, players activated an AR animation of Mr. Monopoly, along with financial literacy tips and clues to fond more squares. Players could also enter for a chance to win prizes, including up to $50,000 in cash and a Jeep Wrangler.

“It was very, very sticky. Of the people who signed up to play the game, 80 percent finished it,” says Jeremy Patuto, principal and chief executive officer of digital agency Gramercy Tech, which collaborated on the campaign with ad agency Anomaly and production and entertainment content firm m ss ng p eces.

He attributes this to making the game both user friendly, with no need to download an app, and simple. To help players find a Monopoly piece at the Flatiron Building in New York City, for instance, the clue read, “Find the building that is flat.”

“We let everyone have fun and experience AR in its fullest form,” says Patuto. “The technology may be complicated, but the experience shouldn’t be.”

Even with the promise of more-immersive AR, Matt Maher, founder of M7 Innovations, a tech consultancy focused on immersive realities, says that “driving sales should be the focused use of AR.”

“As AR grows, every element of the physical world is going to have a virtual counterpart or its ‘digital twin.’ Brands should start thinking about what digital experiences they can provide for consumers around these physical touchpoints,” says Maher. “Concurrently, for any brand that sells a product, AR transforms more than a billion smartphones into virtual storefronts, allowing users to try on and try out their products.”

Even food. In its third AR filter for fast-casual restaurant chain Panera Bread, M7 Innovations worked with AR and 3-D modeling firm QReal to create a mobile ad unit, using both the front- and back-facing cameras, that entices users to “experience” an animated Panera breakfast.

After being prompted to “yawn,” the user sees a virtual steaming cup of coffee and a rising sun appear in front of her or his face, before flipping to the back camera for a view of a full Panera breakfast—the cup of coffee, a breakfast wrap, a napkin and a vase of flowers, all presented on a serving tray.

The ad was deployed on both Facebook and Snapchat, and Maher says the results were “exactly” what were hoped for. On Facebook, 25 percent of users who clicked through and engaged with the AR ad followed up with a café visit. On Snapchat, 2.8 percent of users who swiped up made an in-app purchase with Panera after engaging with the ad. In total, it reached 9.3 million users on a budget of $50,000.

While Maher is bullish on AR, he cautions that some effects may be dead ends for a lot of brands. For example, Snapchat has a Sky Segmentation feature that can “turn the sky into the equivalent of the largest IMAX screen ever. But playing the latest Panera spot in the sky probably isn’t going to drive café visits,” he says. “Still, we are always looking to see what the next move will be, but we’re only diving in if it makes sense for the brand.” And figuring that out is part of the challenge. ca