In my time at the MIT Media Lab, my research focused a lot on experiential or “immersive” theatre. I’m talking about authored narrative experiences that take place in the physical world; think of works by the likes of Punchdrunk, Blast Theory, or Nonchalance.

While most of these aren’t necessarily “games”, it is really interesting how they use their game-like mechanics and interactive systems to reinforce the theatrical elements of their stories. The way they take various “gimmicks” and craft experiences around them that both support their interaction patterns and enrich the work reminds me of narrative videogames in the best possible way.

I want to reflect for a bit on a few pieces that do a particularly good job of marrying mechanic and narrative, in ways that highlight what’s truly beautiful and unique about this sort of work.

Sleep No More’s Mask

Punchdrunk’s Sleep No More is the poster child for what I’m talking about. It’s been around for over a decade; the current New York production has been running for over five years with no sign of stopping. It’s a juggernaut, and many people’s first (and only) exposure to this sort of work.

For the uninitiated: it’s Macbeth, mixed with Hitchcock’s Rebecca, by way of a macabre 1920s hotel. You’re set free in a designed space that spans a full cross-town block and five or six stories tall. Actors are also wandering the space, acting out the plots of Macbeth and Rebecca as stylized, largely voiceless dance. You’re free to follow characters, explore the space, or just exist.

One of Sleep No More’s most iconic images is the white audience mask. Upon entering, you’re told to wear the mask at all times, and never speak. “We like you better that way”, a hotel staff member whispers glibly. It seems like a small thing. But the mask is a core part of the experience, and a delightfully efficient design choice.

On a functional level, it lets audience members distinguish at a glance who is a performer and who is an audience member. It reinforces that you’re not supposed to speak, which would ruin the experience for others. It also boosts the work on an artistic level: as a story about characters haunted by ghosts both metaphorical and real, watching the performers be followed and observed by white-masked ghostly figures is a visually stunning image.

That’s to say nothing of the transformative role of maskwork. Masks have deep historical significance, spanning back thousands of years from theatrical forms like Commedia del’Arte all the way back to the earliest human civilizations. Across the globe, traditional forms of worship include rituals where putting on a mask is said to cause you to become the literal embodiment of the spirit inhabiting the mask.¹ Putting on the Sleep No More mask aids audience members in the transformation from their outside selves to playing the role of one of the many ghosts inhabiting the McKittrick.

This all plays into the experience of attending Sleep No More. Whether you’re consciously thinking about this or not, the mask as an object shapes your relationship with the space and the piece as a whole. It affords silence; it affords a different relationship with other audience members than the performers; it affords emotionally leaving the real world behind and immersing yourself fully in Punchdrunk’s world. The mask matters.

Janet Cardiff’s Binaural Soundscape

One of my absolute favorite things to do in New York City is Janet Cardiff’s Her Long-Black Hair, a site-specific audio walk through Central Park. Originally commissioned by the Public Art Fund in 2004, Cardiff’s piece takes you on a meandering 25-minute journey through the lower half of Central Park, transporting you to other worlds through a combination of interesting stories, thoughtful prose bordering on poetry, and a stunningly beautiful and subtle soundscape.

Thanks to the archival work of Dan Phiffer, you can download the mp3s and photos yourself and experience the piece whenever you want. It’s easily one of the best ways to spend half an hour in New York on a sunny day.

The piece is full of unbelievably clever design by Cardiff and her partner George Bures Miller. I could go on about it for ages. But there’s one particular choice I want to poke at.

A defining characteristic of the piece is its binaural soundscape, and the way it uses 3D positional audio to blur the boundary between the fabricated and the real. Off in the distance behind you, you’ll hear swans honking, or a siren blaring on 59th St, and not be 100% sure whether it’s real or merely an auditory illusion. The effect is stunning.

Because the piece is just a static audio file, though, the sound is hard-coded to be “behind” you. If you turn your head to look for the source, the sound will rotate with you, completely breaking the illusion. This is a pretty serious technical limitation, and one that even today is hard to fix without a head-tracking setup in a controlled environment.

Miraculously, though, the illusion doesn’t break down. At its core, the piece is a poignant exploration of the pain and fruitlessness that can come from lingering in the past. Whether Cardiff is retelling you the story of Lot’s Wife, or explictly telling you that “one of the rules today is we can’t look back”, the piece is directly and indirectly designed around building an environment in which the player doesn’t want to turn around and look behind them.

Not only is the illusion left unbroken, but the player doesn’t even realize anything special has happened, as everything that creates and comunicates that rule/mechanic is also an integral part of the artistic expression of the piece. Even repeating the experience multiple times, with full conscious awareness of what’s going on, the psychological tricks still take hold.

The Grand Paradise’s Doors

This article started brewing in my mind in the days after I saw The Grand Paradise, the latest offering from Third Rail (the New York-based company also responsible for Then She Fell).

Sleep No More is defined by its sandbox-style environment, granting the player a large sense of agency within the physical space. Then She Fell goes to the other extreme, largely removing autonomy in exchange for more frequent and memorable moments of intimacy between player and actor. The Grand Paradise splits the difference, alternating fluidly between more intimate guided experiences and periods where you’re given free exploration of the space.

To make this work, Third Rail uses a very clever conceit: you’re told at the start of the show that you’re not allowed to open any closed doors. Actors and stagehands open and close doors as the piece progresses, dynamically reconfiguring the space as dramaturgical needs demand.

The mechanic works stunningly well within the context of the piece. As a work of art, The Grand Paradise talks a lot about the passage of time and the choices we make with the limited time we’re given on this earth. The metaphor that there are always some doors that are open to you and some doors that are be closed to you, and that any given door may open or close over time, fits in effortlessly with the shared imagery and suggested themes that make up The Grand Paradise’s artistic vocabulary.

At first glance, this is exactly what I’ve been talking about. Like Punchdrunk’s white masks or Cardiff’s insistence that you not look back, Third Rail’s doors effortlessly align the design affordances and constraints of a specific physical interaction ruleset with the sorts of emotions and artistic ideas found in the text and choreography of the piece as a whole.

Here’s what baffles me, though. This “closed door” rule is communicated to the audience in the pre-show briefing, presented as part of an airplane in-flight instructional video. It’s mentioned in the same breath as telling you to turn off your cell phone and the legally-mandated disclaimer that you should probably know where the fire exits are in case anything lights on fire.

It’s framed clearly as a game mechanic, a non-diegetic rule imposed arbitrarily on the world of the Grand Paradise. It exists as an external force imposed purely for the sake of logistics, rather than being a natural emergent property of the world.

This “closed door” rule is super smart. It solves a very difficult design problem, and does so in a way that’s aligned with the piece’s artistic goals in the same way that Punchdrunk’s masks or Cardiff’s theming are. And yet Third Rail very consciously keeps it separate rather than more overtly embracing that synchrony.

As an outsider not privy to Third Rail’s design process, I found myself wondering about a version of The Grand Paradise where the world itself was more consciously aware of the existence of the doors, the same way that the witches in Sleep No More can “see” the ghost audience while Macbeth and Duncan remain oblivious.

An emerging design space

Digital game designers talk a lot about marrying mechanics and narrative. It’s all the more important in this sort of experience design, where you’re not just designing abstract rules to be enacted by a computer or human players, but physical interactions between players and tangible objects or even other human beings. These sorts of interactions are what enable experiences like Sleep No More to be that much more memorable than either traditional theatre or pure digital experiences.

Physical experience design has been around for a long time. The likes of Punchdrunk and Blast Theory have been at it for more than a decade, and related fields like theme park and haunted house design go back at least half a century in their modern form. Lately, this sort of experience design has been exploding, though, with more and more pieces like these gaining popularity and mindshare.

If the emerging space of experience design is going to keep growing and changing – and while it may have been around for decades, it’s certainly undergoing a renaissance right now – I’d love to see as much discussion around this sort of design work as is happening in other spheres. Creating a large-scale design community around work that is both ephemeral and geographically distributed isn’t an easy task, but an important one.² Talking about how to accomplish that is its own large discussion, but, selfishly, part of my hope for writing this hopefully-not-too-rambling article is to try to contribute to that conversation.

–