When augmented reality (AR) is being rolled out in midmarket hotels, you know it’s made the leap from pioneering sectors like gaming to the mainstream market. That’s what happened last year when hub, the technology-led subbrand of the British hotel chain Premier Inn, began using AR wall maps in its four London hotels. It’s a no-brainer, really, an original way for guests to learn about the area they’re in—they simply download the hub app and tap area bubbles overlaid on their handset screen’s map for more information.
But it’s not only the hotel industry that recognizes the value of augmented reality. In the cultural sector, AR and, increasingly, mixed reality (MR) are being used to enhance the museum experience and engage visitors with content in new, innovative ways. The technologies are in their infancy and shrouded in secrecy, but David Sheldon-Hicks, cofounder and executive creative director of London-based creative studio Territory—which leads the field in creating immersive digital experiences and virtual worlds across film, graphics, visual effects, gaming and broadcast—is enthusiastic about the possibilities these technologies offer museums. “The beauty of AR and MR is that the technology is far less an intrusive experience than virtual reality,” he says. “They blend reality with created reality and content in a way that can be very powerful. This is particularly relevant to museums and exhibitions, as the ability to take physical objects and places and augment them with additional graphics and animations allows a blend of two moments that can transform a person’s understanding of a concept.”
A powerful example of this already captivates visitors at the recently opened Centre International d’Art Pariétal Montignac Lascaux, in the Dordogne region of France. Designed by UK creative agency Casson Mann, the center walks visitors through a representation of a nearby historical site that has been off-limits for years: the caves of Lascaux. Discovered by four boys in 1940, these caves are covered with stunning wall paintings of animals that roamed the area some 17,000 years ago. The new center offers a near-perfect facsimile of the cave system, and it further engages visitors with complementary content. “It allows visitors to contemplate the artwork up close, but also feel like they’re part of the surroundings via a range of interactive interpretation aids [spread] through four different spaces,’ says Craig Riley, a director at Casson Mann.
To create a narrative, Casson Mann has used a combination of AR and virtual reality (VR) on both a handheld device, called a CdV (companion de visite, or “visiting companion”), and 3-D stereoscopic glasses because, as Riley says, “both methods have a relationship with what the visitor sees in front of them. The combination allows interpretation without having labels or physical graphic panels cluttering the space and getting in the way of the experience.”
“[The project] presented many challenges,” says Dinah Casson, cofounder and creative director of Casson Mann. “But perhaps the greatest was the search for ways to take visitors away from their smartphones and to allow themselves to be washed with the glory and mystery of the images in the cave: in essence, to remember the power of the unedited, unfiltered experience.”
It’s this need to retain the strength of the content while creating a better way to engage with it that appeals to museums and curators, for whom one of AR’s clear advantages is the ability to load different content onto a device for different users’ needs. This is the case at Lascaux, which offers different modes for accessibility needs as well as adults and two ages of children. And it’s not a one-way street. The European Union–funded CHESS project (Cultural Heritage Experiences through Socio-personal interactions & Storytelling) creates nonlinear, bespoke experiences by matching visitors to a range of predetermined personas. Crucially, story lines can be adapted to respond to visitors’ preferences and interests, which can inform future curation of spaces and exhibits.
As James Thomas, managing director of the London studio of creative agency Bond, explains: “Digital experiences provide the curation staff with direct feedback—highlighting where visitors went [and] which content was most liked, consumed, skimmed and ignored. Combining this with visitor comments, feedback and thoughts on social media, designers are able to provide the curation team with interactive dashboards that highlight the success of the exhibition’s impact and reach. This information not only helps curation teams to engage better, but also allows them to react in real time to a visitor’s engagement experience.”
Thomas has been working with Bond’s Helsinki office on an exhibition at the Design Museum, Helsinki, titled the Renewed Collection Exhibition. Telling the story of how design was used to help develop Finland’s modern welfare state, the exhibition uses a range of digital tools to display materials that previously weren’t available to museum visitors, including the more intangible museum artifacts, such as historical events and narratives. Thomas believes that “using tools in the right way enables the curation staff to deliver a balanced experience.” But he cautions overreliance on these new modes of engagement. “Most visitors who enjoy cultural experiences are led, to an extent, by the storytelling of the curation staff,” he says. “I believe the technologies in this space should be used to complement that storytelling and not overpower it; this way, the visitor’s imagination is left to explore both the physical and digital worlds. If the technology is used to enhance the physical exhibition, to bring to life the story being told, then I think [it] is the correct balance—rather than using technology for its own sake, which can be restrictive.”
In the Netherlands, experience design and production agency Tinker Imagineers leads in enhancing physical exhibitions with AR and MR. Michel Buchner, creative technologist at the company, led a VR project in 2016 as part of an immersive experience called the Nest for Nestlé. The experience immersed visitors in a VR world that utilized a wide range of digital media to create both a historic time-traveling story and a large interactive table where visitors could explore current issues in food production. Eye scanners, VR glasses and Microsoft Kinect games were just some of the digital elements used, along with the VR elements. Of these latter, Buchner says, “The technology develops so fast that the production would be run totally differently today.” More important than the technology is knowing what to do with it as it’s developed, he insists. “As the experience worldwide grows, it’s more a matter of reading the literature and doing a lot of testing,” Buchner explains. “And asking yourself what it does to the viewer. And making the distinction between the wow of the first VR/AR experience and a meaningful, well-orchestrated experience. Technology is crude at the moment, but once it gets better, it will be more convincing. The more convincing, the more tangible it will be. MR will change the way we learn, give information and design.”
AR is currently in an unfocused phase, with a proliferation of platforms—such as Microsoft’s HoloLens, Google’s Tango and future iterations of Google Glass—making it hard to predict what will eventually lead the field. But Buchner believes that as mapping platforms (Aurasma and Tango tablets and smart-phones) and associated plug-ins (GuidiGO Studio and GuidiGO AR Composer) are refined, museums will be able to use AR to offer much more intelligent engagement with content than VR offers. “With VR, you can create a world that is closed, but will let you experience Jurassic Park,” he says. “But with AR, you can have a live dinosaur
in the museum hall, in relation to the real skeleton on display. Present space and virtual space are combined.” In this way, it would be easy for a museum to offer visitors things like a Pokémon GO–style treasure hunt through the gallery, an exhibit in its original context and virtual characters moving around in front of the visitor.
Territory is currently experimenting with HoloLens and other devices. Sheldon-Hicks says, “For a studio like ours, with an interest in mixing graphics with photographic imagery, this new paradigm becomes exceptionally interesting and appealing. To place design, art, animation and information into the real world and for the virtual to interact and respond to the real world is like nothing else. The new opportunities and exploration for creatives are very exciting. [But] I don’t see [AR] replacing any technology or mediums; it will find a new space of creativity to inhabit.”
The US-based startup Magic Leap is also on Territory’s radar. “I’m fascinated by what they’re looking into, as the promise of its technology—a head-mounted virtual retinal display—looks incredible … despite the mystery and secrecy surrounding the project,” says producer Alice Ceresole, who joined Territory from the Light Surgeons, an East London cross-disciplinary media arts practice. But she also sounds a cautious note. “It’s important that the technology is used for the right reason, and a good collaborative relationship between curators, exhibition designers, creative technologists and content producers should take that into consideration,” she explains. “After that, it’s about how the technology and content support the exhibition narrative or story point. Sometimes the use will be functional and explanatory in the same way as a reference book or a diagram can be, but perhaps we can use it like other art forms, to inspire and help people experience the world in a new way that only this new medium can make possible.”
It’s impossible to predict just what the medium will make possible, but that’s what’s exciting about it, says Buchner. “The whole of society is changing into digital communication. We prefer to find our partners based on logarithms, [rather] than a standing pitch in the pub. Is it good or bad? I don’t know; it’s just an effect of the changing times.” But there’s one thing, he says, that won’t change. “Human curiosity. It will be our job to stimulate the imagination of our audience … because not all AR/VR representations need to be realistic. You can still make suggestions and ask [spectators] to fill in the gaps and have their own interpretation.” ca