Storytelling in VR

At Talespinners we’ve worked on a variety of different VR titles now — most, alas, still under wraps. I recently gave a talk on some of our resulting conclusions about narrative design for VR. I thought I’d set some of those conclusions down here.

Much of my approach to storytelling in VR borrows heavily from the work I’ve done in other mediums, particularly live events and theatre. A lot of it comes from my work in horror, particularly when it comes to embodiment and empathy.

I should point out that most of my experience is with head-tracked systems like Vive, Oculus, and PS4, alongside a controller for each hand, rather than with mobile VR. Your mileage may vary.

This article is targeted at people from all sorts of backgrounds — so forgive me, game devs, if in some places I’m covering what appear to be basics!

The Rules of VR Storytelling

There aren’t any.

We’re all just making it up and trying out different things. No-one knows the rules at this stage — don’t let anyone tell you they do. It’s a new and evolving medium, the language of VR is yet to mature, and the audience is yet to become familiar with that language.

That said, what I’m going to set down here are some techniques which have worked for us, and which I’ve observed in other titles. Take everything I put down here with a caveat — this is my experience, this is what I think seems to work; there is no one true way.

Storytelling is Design

I see design and storytelling as fundamentally interwoven for VR. I don’t think you can go away and write a story, and then ‘make it VR’ in an effective way.

I’ve seen this sort of story-first approach when people from a TV or film background engage with VR, games, or transmedia. There’s a tendency to figure out what story they want to tell, and then add a layer over the top which is how the player will engage with it. I think this leads at best to voyeuristic experiences that have the player as a powerless observer; while these can work, they often lead to static presentations that miss a lot of what makes VR unique.

For me, the design and narrative are fundamentally interwoven — as in most media, but even more so in VR. So I’m going to talk about design as much as storytelling.

I’d advocate thinking about the player’s role first, and then designing the experience around that.

What’s the player’s role?

The player is present in the VR world. They are immersed in it, it is around them, they are a part of the action. This is what VR does well, and it’s a shame to ignore it.

While they can simply be an observer, I think this is underserving the medium. Even if they are a floating observer in the world, there should be some thought given as to how their presence affects the story, or at least how it might affect their own interpretation of it. Do they learn more about the story because they can see it from different perspectives?

Or can they interact with the world? Can they affect how the story plays out? Are they a character in the story themselves? Is the story fundamentally focused on the player; are they the protagonist? What experiences do you want them to get out of it?

My first step is always to focus on that player’s presence within the world, and ideally to find them an active role in it. This doesn’t mean that you need choices and a branching story; just that the player has some part to play in moving themselves forward through the experience.

Anecdotally, there’s something about being an active participant in a story that makes it more intense; that makes the experience stick with you. Consider the difference between watching someone walk down a corridor in a horror movie, and having to walk down that corridor yourself.

Player Presence

VR can put you in other people’s shoes. It’s great for giving you someone else’s viewpoint, putting you right in the action, and promoting empathy, at least in the sense of ‘Oh my God, this is what it’s like to stand here!’ To heighten that, it’s worth thinking about how to enhance that player presence; to make the player feel as embodied as possible in the world.

Partly that’s about eliminating friction, particularly in game controls. When the player tries to take action, they shouldn’t need to stop and think and then fiddle with buttons that they can’t see. You want intention to immediately become action — “I want to do this” becomes “I have done this”, using the normal rules of the human body, rather than “I want to do this” -> “how do I make the game do this?” -> “what button was that again?” -> “Oh, okay, now I’ve managed to do it”.

This is just good UX and control design; I’ve seen VR experiences with touch controllers where you have to double-click to pick up an object, then click another button to view it properly, and then it sticks to your hand until you click again… and all sorts of variants of that. Humans already understand how to grab, and the controller supports it, so stick to that!

You want to design your control systems to be as closely mapped to what the player already understands as possible; aside from removing friction, it means you can get away with a minimum of tutorials. VR gives us a good analogue to hands; let’s use them. Want to support more actions than are available using hands? Then give us tools to pick up with those hands, or VR buttons to press, or levers to pull; that’s what we’d do in the real world.

I said earlier that people are not yet used to the language of VR. However, they are used to the language of the real world, and that’s what they try to use; so any deviation from that is initially confusing until people ‘get it’.

We have more than hands available, of course — we are tracking the player’s head. Obviously, we can use that to look around; but how about nodding or shaking your head, as in Fated: The Silent Oath? Or heading a football, like in Headmaster? Or ducking under a beam, as in Box VR?

Feedback is incredibly important. If you do something in the world you should expect visual feedback. Audio is just as important; there are very few silent actions in the real world. But don’t forget that VR controllers have vibration feedback. This is incredibly powerful; it can make you feel like you’ve put your hand on a surface, or your sword-tip has cut an object, or you’ve punched a target, or your bowstring is taut. Use it for hand-related actions, even in tiny, minimalistic ways. The player will soon edit out these as vibrations at all, and will interpret them as if they are truly sensing the world; they will believe they are interacting with solid objects. The player’s brain is your most powerful tool.

Proprioceptive Alignment

Okay, I pointlessly used a long word. Proprioception is the human sense of where their limbs and body are relative to each other at any given point; you can close your eyes and ‘feel’ them. Yes, we do have more than six senses…

If you are going to render the player’s body in the virtual world — and you don’t have to! — then do whatever you can to make sure that the poses of the body are aligned with the player’s real body. This could be as simple as asking ‘are you sitting or standing?’ at the start of the experience. If the player’s avatar is doing something drastically different from what the player is really doing e.g. lying down vs. standing, the cognitive dissonance separates them from the experience, and makes them think about the difference between the real and virtual worlds, whereas we want to do all we can to eliminate that difference and immerse them completely.

A fantastic example of this kind of design is the demo experience Kitchen that Capcom put together in the early days of the PSVR. In it, you are sat on a chair in the middle of a horror-filled kitchen as someone stalks the space around you. It’s an experience that’s designed to be used with a standard PS4 gamepad — which means the player is sitting down, pad held in both hands. Taking this knowledge, the demo’s creators decided that the player’s avatar would be tied to a chair, and that their hands would be tied together in front of them — which is the exact pose that the player was already in. And because the PSVR can track the gamepad’s location, whenever the player moved the pad, that avatar’s hands moved in response. As a result, player and avatar were near-perfectly aligned. A very simple idea, but extremely effective.

Another way to promote this sort of immersion is to use the VR rig’s reality as your game metaphor — you are strapping a device on with a restrictive field of view and holding something in each hand. For example, pulling on a diving helmet in Subnautica, or a spacesuit in ADR1FT, or Batman’s mask and gloves in Batman: Arkham VR.

Delivering Story

So with the player immersed in your world, how do you deliver story to them? In a way the question answers itself; the VR experience is all about that new world that the player has stepped into. They’re through the door and now they’re in Narnia. That’s fundamental to how we deliver the story; to treat the experience as a living, breathing world, and, crucially, to allow the player to discover the story at their own pace.

This is partly because of where we currently are in the medium. Many players, on first putting on a headset, are still sufficiently new to this sort of experience that they’ll spend ages looking around in awe or trying to pick up things to see what they can do with them. If you’re trying to force a story on to them while they’re going through this process, you’ll lose or confuse them. So let them explore, let them experiment, let them pick things up and fumble their way forwards. Let them discover pieces of your story by themselves.

The games industry has lots of great examples of stories told purely through objects and environmental storytelling — games such as Gone Home. A lot of the principles used for that sort of game can be applied very effectively. There are restrictions and differences, of course; notably, text is difficult to read in VR and there needs to be careful thought around the design of readable objects in-game.

This is not to say that a VR experience can’t have a core game loop, such as shooting, or fishing, or driving, or defusing a bomb; of course it can. I’m just laying out here one way of thinking about how to deliver story.

Story Through Exploration

A few simple examples of tools we use for storytelling in VR:

Points of Interest

These are things which will attract the player’s attention. They could be items to interact with (letters, buttons, levers), or particular objects to reach (the altar in the temple, the door), or characters (a face looking in at the window). They are all things designed to draw the player’s eye and to pull the player’s thoughts towards them.

We call attention to them with strong or distinct visuals, or careful environmental design, or specific sounds. The player could learn story simply from passively studying the object — like reading a signboard, or looking at a chalk outline on the ground — or could interact with it, to trigger something.

Triggers

Triggers fire when the player performs a specific action. A trigger might be an area of the world that the player steps into (they step through the door into the temple), or a particular action they perform (pull that lever). Normally they’re tied to a point of interest, or triggered by trying to get to one. They are all caused by player actions.

The result of a trigger is usually to have the experience respond in some way; a sound played, a character speaking, an animation playing, the player teleported to a new location, a set piece kicking in.

Set Pieces

Set pieces are canned scenes of content which play out. They are analogous to cut-scenes in traditional videogames; however, unlike cut-scenes, the player still has some amount of control during them. At the very least the player still has the freedom to move their head and hands around.

Set pieces could be background characters having a conversation, a pair of huge doors opening, the train that the player is on setting off; any vignette which lets the player observe, interact with, or discover more about the story, changing their world in some way.

Breadcrumb Trails

We build up the experience by laying out breadcrumb trails of points of interest; as the player explores them and interacts with them they fire off triggers and set pieces, which move the story on. Critically, most of the time the player is driving the story at their own pace. This is very similar to typical computer adventure design.

Mood

You can tell layers and layers of story through the world around the player. The design of the environment — art, lighting, effects, sound — all come together to create a mood for a scene. Of course, this is true of cinema, video games, and theatre; but in VR the player is inside this scene. Again, anecdotally, they are much more affected by their environment as they are so immersed in it.

Space and Scale

VR is all about space, something which doesn’t really work in the same way as it does in film or TV. Because you’re standing inside that space, how it’s architected becomes terribly important. You can tell story using cramped spaces, oppressive spaces, open spaces, spaces that guide the player, spaces that interrupt the player.

This has a direct relationship to scale; in VR, giant monsters really feel as big as skyscrapers; or you can really shrink down to the size of an ant. Giant Cop is a fantastic example of this. And you can use scale to tell story; there’s a moment in Batman: Arkham VR where (spoiler!) you are young Bruce Wayne in the alley behind your parents, just as they are about to be attacked. You are a child’s height, and your parents tower over you. You peer out from behind your mother’s legs to watch what’s happening. You feel small and helpless.

Talking of space, you can also tell story through emotional, personal space. You can design the space between the player and other characters to tell stories; at a distance, people are strangers. Up close, they are friends, lovers, or enemies who are violating your personal space; this is a language that people understand on a subconscious level.

Characters

Characters are incredibly important in VR. Even if they have low-poly graphics, even if they don’t obey the laws of physics, our hind-brain is designed to find characters in an environment (take a look at the phenomenon of pareidolia if you don’t believe me) and, if you give the brain something which looks like it could be alive, moves as if it has motivation, and give it a voice, the player will subconsciously believe that the character is ‘there’.

Characters have presence and vitality in VR in a way which they don’t on a flat screen. They feel alive. So you can use that to tell story in all sorts of ways, using mechanics that the player innately understands — body language, eye-lines, emotional distances, and above all, voice.

What VR Isn’t

I briefly want to cover some of the things that VR isn’t, from a storytelling point of view.

VR Isn’t Cinema

• You can’t control the player’s head, so you can’t frame your scenes in a cinematic way. You can use set design — colour, shape, light — to guide them, but you can’t grab their attention and make them look at a particular object by moving the camera.
• Similarly, because you aren’t using a camera, you can’t use tricks such as playing with depth of field or using a focus pull or zoom in and out.
• Cuts are an issue. When you use a cut in VR, you are interrupting someone’s experience of living, and we’re not used to that. It can work if you’re careful when you’re moving from scene to scene, but the sort of back-and-forth cutting used in cinema just won’t work.
• The player is in the scene, rather than watching it.

VR Isn’t Games

• Many games borrow heavily from cinematic language, particularly in visuals and cameras, and we can’t use a lot of that — see above.
• Many modern games are built around the idea of locomotion; that you are traversing a physical space. For reasons of motion sickness, this isn’t a great fit into VR — there are ways around this, but it’s best to be planning for VR first rather than taking a more usual game level design approach.

VR Isn’t Theatre

• Actually, I think writing for VR is closer to writing for theatre than any other medium. Specifically, I think it’s closest to immersive theatre for an audience of one!
• Even the best immersive theatre — work by Punchdrunk, for example — rarely allows much audience agency, and often treats the audience as observers. VR can do a lot more than that. But there’s a very good example of what’s effectively an immersive theatre story told through the medium of VR — check out The Invisible Hours.

VR Isn’t Realistic

We’re still in the early days here; but many people are already chasing the Holy Grail of fidelity — amazing-looking realistic imagery at very high frame rates. You don’t need to do that. Humans are perfectly happy to fill in the gaps themselves; stylisation is perfectly acceptable. So long as you tell an engaging story with engaging characters, you can dial back on the realism; especially given that however realistic the rendering, we’ll still be able to put our hands through walls.

Think Pixar; think The Muppets.

Read Scott McCloud on how we understand broad-brush cartoon faces much more easily than, say, the Mona Lisa.

When it comes to acting, we aren’t going to be able to read nuanced facial expression, even if it’s well animated or motion captured — the fidelity just isn’t there and we won’t have any closeup shots. It’s something to think about when hiring mocap actors for VR — you might want to look at people who can handle a theatrical style, as they know how to sell body language to be read from a distance.

What VR Is

At its root, as I’ve said a few times in this article, VR is stepping into another world, something no other medium can do as effectively.

In Summary

• Storytelling and design are fundamentally interwoven for VR.
• Concentrate on the player’s role in the world, and build the experience and story from that.
• Remove as much friction between the player and their avatar as possible — make the controls frictionless and give them feedback.
• Let the player discover the story at their own pace.
• Use the things that VR can do well — player presence, immersion, tricks with size and scale, that sense of life inherent in other characters.
• Focus on the broad strokes instead of realism.

Give them a world to step into!

Examples

Finally, some examples to take a look at:

• Lone Echo, which I think is the best full storytelling game on any VR platform right now. It absolutely nails so many of the things I’ve mentioned above.
• The Invisible Hours, as an example of immersive theatre on VR.
• Rick and Morty: Virtual Rick-ality as a great example of playing tricks with space and scale.
• Fated: The Silent Oath for an interesting take on interactions, including head nodding and shaking.
• Skyrim: VR for one of the video games which actually survives a translation to VR extremely well.
• Capcom’s Kitchen for a great example of proprioceptive alignment… uh… putting the avatar in the same pose as the player.
• Batman: Arkham Asylum for some great storytelling, some clever thought about adapting to the limitations of VR, and characters who you feel are in the room with you.
• Giant Cop: Justice Above All as a brilliant example of scale!

Thanks to Matt Gibbs and Rachel Thomas for feedback.