Human Communication

Around the world locations are becoming new places to perceive our everyday surroundings differently via mobile phone-based augmented reality (AR). Personal computing is migrating from the one-way 2D screen to the mobile camera lens/3D view, promptly changing the rules of UX/UI design.

Where will you anchor digital content? What are you going to make your users feel? How does the AR designer respond?

All the world’s a stage

Architectural, product, or UX design methodologies alone are not enough to deliver great AR immersive experiences.

AR immersive experiences are a type of storytelling were design should strive to serve the story, whether the work of a single visionary writer or local individual stories. Attention to design forms part of the reason why Samuel Beckett and Henrik Ibsen are story telling master innovators.

‘Seeing place’ is the etymology of the word ‘theater’ stemming from the ancient Greek ‘theatron.’It is theater design history that serves augmented reality.

1. Define the Mise-En-Scène

Mise-en-scėne referes to “everything placed on the stage or in front of the camera — including people. A catch-all for everything that contributes to the visual presentation and overall look of a production.” When translated from French, it means ‘placing on stage.’

Acknowledging the size, shape, and layout of an indoor/outdoor space directs a performance’s mise-en-scène, the design aspect of theater production. It is the visual theme or telling of a story.

On entering a theater of any kind, a spectator walks into a specific space, one that is designed to produce a particular reaction or series of responses, the reception of that space becomes part of the total theatrical experience. What will the AR experience provoke?

Nobel Laureate for literature Samuel Beckett preparing ‘Waiting for Godot’ for theatre © Gallica

2. Use Proxemics

The Apple AR(t) initiative is an iphone guided walking tour at six cities includingartists Nick Cave, Cao Fei, and John Giorno. Similar to a scenographer (theater designer), as part of this design process, the artists applied;

Proxemics — study of how humans use space when communicating, the manipulation of space and spatial relationships among people, (digital) objects, and setting/location.

Where to anchor content? What scale should the digital content be at this location? How should this make the user feel? By answering such design questions, combining unique location, digital content, and storytelling, AR experiences can avoid becoming stagnant and billboard-like.

3. Use Composition

Being the pleasing arrangement of actors/digital content within the theater ground plan/map of the location for the AR experience that lets the audience/users know where to look at a given moment.

Drafting a ground plan indicates the proxemic potential of the actors/digital content and the theater space. The purpose is to discover dramatic actions and to illustrate these in the simplest possible way through emphasis or contrast.

Composition does not require movement at all, it is a static caught movement in time and space, leading the audience to see what the director/AR UX designer believes is essential. It concerns itself with clarity like a still life painting of the stage at one moment in time.

4. Use Blocking

“The theater demands democracy for the same experience from every angle,” — Es Delvin, international stage designer

Blocking is the choreography of the play/AR experience. It is clarifying the movement of the actors/digital content or users and the unique relationships of the characters.

The physical distance between people can relate to social, cultural, and environmental factors, even stressing character and plot development. Something as simple as strategic placement, lighting, and controls can make an emotional impact on the user.

Es Delvin, international stage designer ©The Face

5. Use Picturization

A visual interpretation of each movement so the dramatic situational nature is covered to the audience/users without dialogue or action. The process meaning, subtext is added, the storytelling aspect of blocking, that which is not supported by words.

“I am not only designing what the audience can see on stage but also the empty space that they can’t see.” — Minglu Wang, scenographer

It allows the designer to explore and discover. In Japanese theater, ‘Mie’ or ‘frozen moment’ is a powerful and emotional pose struck by an actor who then freezes for a moment. It is meant to show a character’s emotions at their peak, one that remains in the mind of the viewer.

6. Collaborate within a Vision

A commonality in pathfinding AR projects is the need to lead a diverse team of collaborators with titles such as subject matter expert, curator, producer, developer, UI, graphic, and 3D designer, field recordist, audio designers, researcher, and narrator.

The skill to lead towards a sole vision, similar to a scenographer who is responsible for all of the aesthetics of a performance, is becoming evident.

7. Design for the Human Locale

“Theater is a ritual, one should go out of the theater stronger and more human than when you went in … epic theater is where the audience shares a moment of consciousness.” — What theater does, Ariane Mnouchkine, Théâtre du Soleil cofounder

The aim of one Berlin based AR app was to “understand what it was like to live in a dictatorship and how valuable freedom, democracy, and human rights are.” A noble design vision don’t you think?

Here’s to the Storytellers & Ephemerality

Staging of ‘The Encounter’ at the London Barbican by Simon McBurney was minimalist. Sound was deployed to aurally immerse the audience in the play’s setting, the Amazon jungle. It exemplifies how spatial design means not simply designing the space on the stage, but the space inside the audience’s heads.

“Theater-makers are aware of the ephemerality of what they’re making. In the end, everything is only going to exist in the memories of people,” — Es Devlin, stage designer

Mobile AR technology enables the creation of imaginary theatrical worlds wholly independent of temporal, spatial and physical constraints. As the script of a play is nothing without the successful execution on stage, an immersive AR experience idea is nothing without the careful direction of design.

Theater design and augmented reality are here to serve the next generation of great storytellers — who always shape society.

First published at VentureBeat, January 2020

If you enjoyed any of this piece please click the follow button and subscribe below.