VR offers additional experiences and immersion, but the technology should not be used just for the sake of presenting content in VR, writes BBC R&D’s Alia Sheikh.
It was a rainy day in York, and getting darker by the minute, but none of this mattered to the woman on the swivel chair. She wore a bulky VR headset and her mouth was a delighted “O” of surprise. She looked like she could be somebody’s grandmother. But in that moment, she was a Viking.
That woman was immersed in Virtual Vikings - The Ambush, an educational piece which won Best VR experience at ASFF, a festival I was on the VR jury for last week. The experience uses videogrammetry to capture human-actor performance for the Vikings, deliberately cinematic lighting, objects and characters who serve to direct your attention and locations that correlate to how the viewer is positioned in the real world. The full museum experience even has ‘4D seats’ to simulate the motion of the ship – a level of immersion that helps you buy into the story environment in a very fundamental way.
What was interesting, however, was that the first time any of the jury tried Virtual Vikings, we were inadvertently played a version that removed all of the characters who interacted with the viewer.
It was fascinating to experience content that emphatically did not work, immediately alongside a version that did. With all of the speaking characters removed and no-one left to acknowledge the viewer, Virtual Vikings became a lonely experience, beautiful, slightly meditative, but with nothing to ground it. The act of covertly observing Vikings going about their business was unable to hold my attention for even a minute.
- Read more: BBC shuts down VR Hub
The working version on the other hand – containing characters who acknowledged you, fought alongside you and died in front of you – was an altogether different experience. The emotional core of the story relied on convincing you to accept an alternate reality in which you were part of a group, meaning you were then emotionally invested when that group was attacked. This observation reinforces earlier findings on the importance of responsiveness of characters and embodiment of the viewer.
Stories should only be told using VR if there is no other medium that affords the particular experience you want to convey. A story can be harmed or outright lost if it is presented in VR unnecessarily. VR’s strength as a medium is that it allows us to immerse the viewer in a plausible reality – one that the viewer intuitively accepts as part of their immediate landscape. When used properly and with compatible subject matter, immersive experiences allow us to remove more of the barriers between viewer and content, and that, in turn, facilitates an emotional connection to the story.
I’m Still Kenny
On the far simpler but equally effective end of the technological scale is An SPMS Story – I’m Still Kenny, a 360 film about the day to day life of an MS sufferer which uses ‘zip-lining’ (unidirectional forwards motion) to move the viewer through a gentle story landscape that is somehow both literal and abstract. It’s an excellent example of an immersive piece which creates a feeling of human connection by carefully presenting one person’s very specific story. The film succeeds in immersing the viewer inside a representation of the illness that, much like Kenny, they cannot escape.
My colleagues at the BBC’s VR Hub have recently wrapped up a two-year portfolio of work and their fascinating report gives audience insights and advice on making a great VR story. The report is grounded in several real-world examples of their work, including immersive journalism projects with BBC Africa correspondent Alastair Leithead on two VR documentary series, Damming the Nile and Congo VR. These were filmed in 360 to not only tell fascinating stories but to help people better connect with parts of the world they’re unlikely to ever visit. Too often news from areas like this can feel an entire world away, but by immersing you in the environment with VR the viewer forges a much stronger connection with the story and the people affected.
Telling stories in VR works best when we are able to conduct the viewer’s attention around the story landscape. However, immersive formats are often most powerful when viewers feel as though they are making discoveries for themselves. One of the first pieces of research our team did was to learn how to unobtrusively direct attention in immersive environments – ideally, the viewer should feel they are choosing to pay attention to whichever part of the story you are currently leading them to.
We also should not underestimate the role of choreography and movement when telling stories in immersive landscapes. We have borrowed the use of movement cues from theatre (immersive theatre in particular) which can be used to lead us through an environment, and, in turn, the story. Movement cues are demonstrated beautifully in Crow: The Legend, which uses techniques such as passing focus between actors and the use of motion to lead the eye.
Conveying depth in immersive environments is straightforward for CG rendered environments, but less so for photo-realistic ones. However, we can envisage applications for photorealistic environments that have depth, in particular for use cases where we want to show something both honestly and as realistically as possible.
Right now at BBC R&D, our team is working on early storytelling research into light fields which are able to recreate the behaviour of light for a range of head positions – this allows the viewer to ’see around objects and corners’. Light fields reproduce effects which we see in the real world, but not in 360 video. We are therefore able to produce environments which appear as realistic as 360 video, but are as responsive (in terms of representing depth cues, parallax, reflections and occlusions) as 3D games. Creating low-cost light fields which afford experimentation is an interesting technical challenge in its own right.
Again, however, the same questions apply – why make content in this medium rather than any other? What is the potential role of the light field as a storytelling tool?
To answer that question, I’ve been looking at the use of the moving shot in traditional cinema. Often used to introduce the audience to a story world, moving shots convey a sense of realism. They provide visual cues which convince the viewer that the story is occurring in a richly textured space, a setting with layers and depth – a place that the viewer can viscerally accept as being real. Moving shots are also used in cinema to promote a feeling of discovery and of progress being made. These observations give some clues as to what types of content will work.
It’s unthinkable that we will not, someday, have easy access to the kind of immersive technology which we can, at present, only dream of. More comfortable, more convincing and more accessible – the future is inevitable.
Our role, then, becomes to learn from our current, imperfect, immersive environments, and to conceptually extend mature art forms like theatre, dance, television and cinema.
By continuing to experiment in these spaces we make it possible to unlock new ways to do the same thing we’ve been trying to do all along – tell each other our stories and make each other feel something real.