Future of Film is in Gaming

In Hamlet on the Holodeck, Janet Murray says that, “The kaleidoscopic power of the computer allows us to tell stories that more truly reflect our turn-of-the-century sensibility. We no longer believe in a single reality, a single integrating view of the world, or even the reliability of a single angle of perception. Yet we retain the core human desire to fix reality to one canvas, to express all of what we see in an integrated and shapely manner. The solution is the kaleidoscopic canvas that can capture the world as it looks from many perspectives – complex and perhaps ultimately unknowable but still coherent” (Murray 161).

Virtual reality filmmaking offers the framework to tackle these new notions of storytelling.  However, VR filmmaking represents a fundamental shift in video making and offers new challenges. David Sheldon, creative director for VR studio Territory says, “360 content and VR feel more like working with theatre than films.” With film, one can edit between shots and focus and light to create beautiful compositions. With VR, editing creates a jarring experience. The viewer also does not have to look where the director wants them to. Attracting the viewer’s gaze to the action means rethinking narrative storytelling in a field in which the tradition cinematographer’s ‘shot’ is meaningless.

Filming in VR offers unique limitations to a director and his or her crew. The camera must be at the appropriate height, so as to not make people appear larger or shorter than they are in real life. It also has to be close to the action, because most 360 camera rigs do not have zoom capabilities. Because a viewer can look in all directions, the lights either have to be well hidden between the seams of the separate lenses or a part of the set. Above all, movement control rests with the viewer. Dan Phillips, VFX firm MPC’s head of digital and interactive, says, “The emergent cardinal rule of VR development seems to be, under no circumstances take the control of the camera away from the viewer – an alteration of viewpoint for someone whose head remains static is an instant one-way ticket to nausea.” Many rigs also do not allow for 360 playback (rather some offer panoramic playback or multiscreen playback. Nokia has a £40k 360 playback functionality for industry insiders. This lack of on-set playback means that directors have a lot less agency to insert themselves in the captured performance.

To deal with many of these limitations, Francisco Lima, VFX technology supervisor at Gramercy Park Studios, advises a “previz/postviz approach” to VR filming. He advises shooting a rough 360 video or the location and then building the VR environment with the DP, production designer and VFX supervisor before the main shoot. He says, “This allows you to plan for lighting and camera positions, or hiding crew.”

More than proper preparation, many studios should turn to the gaming industry, whose experience creating interactive experiences far outweighs the film industry’s current experiments. It’s not enough to get filmgoing into the VR world. You have to give them agency. As Janet Murray says, “Agency, then, goes beyond both participation and activity. As an aesthetic pleasure, as an experience to be savored, it is offered to a limited degree in traditional art forms but is more commonly available in the structured activities we call games” (Murray 128). Even so, the game industry has struggled with the physical limitations of VR. Using the left-stick to provide artificial locomotion still creates motion sickness, so game studios have opted for alternative methods of simulating movement. nDreams The Assembly, Oculus’ Damaged Core, and Bethesda’s VR version of Fallout 4 all use teleportation. The player looks at a space in the virtual world, hits a button, and is transported there. The gaming industry has worked out how to make such movement as transparent as possible. Tomas Gillo, Vice President of development at nDreams, says that, “While we do offer a traditional first-person control scheme, by default players are presented without ‘blink mode’ – a line-of-sight based movement scheme that enables players to transition from point-to-point within the game literally in the blink of an eye. It’s too fast to make a person sick, but just slow enough to register as movement rather than a teleport…This system has enabled us to build a full five-hour game.” Such a system could easily be used by a film studios. Rather than filming a scene from multiple angle, a filmmaker would film an event from multiple locations, and viewer could travel at will along a latticed branching predetermined path through the ‘live’ event (see Oculus Damaged Core for example in gaming). This would allow for what Janet Murray calls “multiform stories.” As Murray writes, “I am using the term multiform story to describe a written or dramatic narrative that presents a single situation or plotline in multiple versions, versions that would be mutually exclusive in our ordinary experience” (Murray 30). Using a 360 headset would enable the viewer to see multiple versions of the same film. Allowing movement would go further and create hypertext narratives, like that of Michael Joyce.  

The gaming industry has further advice to give to enable film to create such paradigm shifting stories. EA bypasses the issue of locomotion by setting Battlefront: X-Wing (announced June 2016) in the cockpit of a dogfighter. Ubisoft has discovered, quite brilliantly, that narrowing a player’s field of vision during intense moments of movement gets around nausea, enabled better, faster, viewer led movement. While Resident Evil 7 (ann. June 2016) and Fallout 4 (ann. June 2016) let players take on more centrally narrative adventures, Star Trek: Bridge Crew (ann. June 2016) and Batman: Arkham VR (ann. June 2016) are far more heavily mediated, less open-world, short-form experiences, more in tune with a feature-length film. Ubisoft’s Star Trek: Bridge Crew is a co-op challenge, where four players take positions on a USS ship. As virtual reality developer, Joe Radak, reports from E3, “I’m super excited for Bridge Crew. I think of all the VR games announced by the major publishers at E3, this one showed the best use of VR and the Rift/Touch capabilities. It’s social, it uses the hand motion controllers very well in ways that you can’t do in traditional media and on top of that, it looks immersive and fun. It showed that there are some AAA companies that really took time to research what works and what doesn’t.”  

While watching a VR film is a very singular experience, VR films that take off of Star Trek: Bridge Crew could bring VR film back to the very communal roots of movie watching. Using the techniques of the gaming industry, entering hyperreality could be as simple as going to your nearest AMC. What was once sublime and cutting edge will become ordinary. And as technology gets better and better, what was once a distant reality, could become the preferred one.

Murray, Janet Horowitz. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. N.p.: Simon and Schuster, 1997. Print.