Keeping it Real

Artists and technologists bring people together in mixed reality experiences with Epson projection

Can a high-tech environment be calming as well as captivating, communal and immersive? Yes! says technology consultant Leila Amirsadeghi. “There’s an opportunity here to create the next generation of entertainment that’s interactive, engaging and social,” she explains. The Unreal Garden, a multiplayer mixed reality environment, merges the strong appeal of online media and video games with face-to-face, person-to-person interaction. The platform it’s based on, which its developers call an example of “social reality,” merges video, sound, augmented reality, and other innovative technologies to create a beautiful, almost magical world. A crucial part of the experience is projection mapping using Epson LightScene accent lighting and Pro L-Series large venue laser projectors.

Building a mixed reality experience

The Unreal Garden, as it was first unveiled, was part of a suite of immersive experiences Amirsadeghi helped introduce at Onedome, a brick-and-mortar entertainment venue on Market Street in San Francisco. Open there from October 2018 to July 2019, it lives on as a traveling experience, which debuted at the E3 Electronic Entertainment Expo in Los Angeles in June 2019. The experience was a reaction to research that shows many users of social media and video games feel isolated, as their screen-related activities take time away from direct, interpersonal relationships. “While we brought The Unreal Garden to E3, it couldn’t be further from the violent games and competitive e-sports that are a big part of the conference,” Amirsadeghi explains. “People at the conference saw it as a calming respite from the chaos of the show floor.”

Amirsadeghi has been involved with immersive entertainment for more than two decades in one form or another. As CEO of consulting firm MESH, she works closely with Enklu, the developers of the platform on which The Unreal Garden was built to continue to drive the future of immersive mixed reality using AR as the foundation. She explains that, using the Enklu platform, The Unreal Garden includes five immersive layers. The first is a physical layer. In the version at Market Street, it consisted of rocks, plants, trees, pagodas and ponds built of physical props with real water features. “At E3, we had a ton of plants, but created those ponds using prop rocks and then used projection to fill in the water,” Amirsadeghi explains. “The second,” she says, “was sound as a layer of immersion.” The developers set up speakers in different parts of the room, with sounds appropriate to the various features, such as flowing water. “But some of the sound was played inside the augmented reality headsets, based on the content you saw.”

The third is a projection layer, adding colors, movement, and interest to the physical layer plus immersive effects. “We used some of the projection in a passive way, adding, for example, koi fish swimming in the ponds. But we also added a layer of interactivity visible without the headsets, triggered by the user and their willingness to work with others within the space.” For this layer, Amirsadeghi and her team used a combination of Epson Pro L1505UH, Pro L1300U, Epson Home Cinema LS100, and LightScene EV100 laser projectors. With the Pro L projectors, they used a combination of Epson ELPLU03, ELPLW05, ELPLM08 short, wide and middle-throw zoom lenses. The fourth is an augmented reality layer, using Microsoft HoloLens headsets that allow guests to see purely digital objects and effects, in three dimensions, as though they were physically present in the room. For example, the Garden included a set of colored glowing orbs floating in the air, each above a pedestal attached to a rock formation. Guests could walk up to an orb and pop it with their fingertips, unleashing a burst of color and sound. A crucial feature of the Enklu platform is that all guests experience the same AR effects through their own HoloLens glasses. Thus, if someone pops an orb, everyone looking toward the orb sees it pop, though each from their own proper perspective, creating a shared experience. The fifth is a haptics layer, a set of effects directed at the guests’ sense of touch. “We used UltraLeap technology, which creates ultrasonic waves to project sensations onto your hand,” Amirsadeghi explains. The pedestals below the orbs each include one of the UltraLeap units, “so if you pop an orb, you get a surprise, your hand meets resistance as though you are touching an object.” “The reason these layers are relevant,” she adds, “is that people connect through shared experiences and shared memories. To bring strangers together, it’s important to create ‘wow’ moments that they can experience together. “If you pop an orb by yourself, you’ll see a burst of color. But as people start to play, they realize, wait, if four people come up at the same time and each pop one of these orbs, it will reveal a large, beautiful butterfly Easter egg projected on the rocks behind the pedestals. “So you see, as a single person, you can have fun, but you can’t help but see what others are doing and want to ask them about it. Once you realize that you can get these great visuals if you work together, you want to do so.”

A F.E.A.S.T. for the eyes, ears and plate

In addition to The Unreal Garden, there were 17 other interactive art pieces at Market Street, most of them based on projection. Fourteen were located in a lower level area called LMNL; another was at the entrance; another in a first-floor café; and then there was F.E.A.S.T., a fully immersive dining experience with interactive effects produced by Epson projectors. F.E.A.S.T. brought 32 people together to sit at one long table to enjoy a five-course meal immersed in visual splendor. “In between the courses we had what I would describe as dinner theater, a beautiful projection show on the table,” Amirsadeghi explains. “As the food was put down, we switched to a set of projections that the people could interact with by moving their hands over the images.” Like The Unreal Garden, F.E.A.S.T. was a social reality experience, seating people with others they had never met. The projections would create conversation points and opportunities to interact. Also like The Unreal Garden, dining experiences based on F.E.A.S.T. will soon be touring other cities, starting in early 2020 and finding permanent homes in location-based entertainment destinations such as the 30,000 square foot facility in Albuquerque called the Electric Playhouse, opening early 2020.

An unreal future

Amirsadeghi says the Market Street venue gave her and the platform developers a chance to try out various types of projection equipment. While the original The Unreal Garden and F.E.A.S.T. used Epson units, the artists featured in LMNL often chose other brands, including lamp-based projectors. From her experience with these devices, Amirsadeghi says she is determined to move forward strictly with Epson laser. “First of all, for large-scale installations, the Epson shortthrow lenses are amazing. With the other brands, we would have to combine several projectors to achieve the same effect we’re now able to get with just one Epson.” The brightness, contrast and the resolution of Epson provides, she adds, is stunning. “In installations like these, it’s really important that the images pop. At E3, for example, we created the ponds strictly with projection, but they looked real.” The Epson laser projectors were completely trouble-free, she says, and could be used all day and all evening without a breakdown or the need to change a bulb. “You can imagine that, in some of the smaller venues at Market Street, the heat from projection lamps was a huge problem, one that we’ve solved now by switching to Epson laser.” Amirsadeghi says she really likes the Epson LightScene, a smaller laser projector built with a form factor similar to a track lighting fixture. Her team used LightScenes throughout Market Street and at E3, and will be using them in future locations. “While we like the bigger Epson projectors for areas where we cover more space, the LightScene is perfect to accentuate a particular prop or design. For example at LMNL, as you came down the stairs, we had this crystal formation. We used a LightScene to make it glow and add movement, to really bring it to life. The best thing about it is that it integrates into the environment, without drawing attention to itself or breaking the illusion.”

In addition to the dining experience about to open in Albuquerque, there is a touring version of The Unreal Garden that will visit several American cities, plus cities in Europe and South America. Other immersive dining venues are planned as well.

“You’re going to see a lot of mixed reality experiences in the future, especially those based on projection mapping,” Amirsadeghi says. “The nice thing about projection is that you can create massive immersions that everyone can engage and interact with, at a relatively low cost." “The biggest thing, of course, is that mixed reality entertainment is social, and it’s fun. I’m excited to be working in the field.”