INTRODUCTION TO 360 VIDEO AND IMMERSIVE CINEMA
There are two ways to create content for virtual reality – computer generated imagery (CGI), or live action, using a virtual reality video camera. The former is generally used for games, the latter involves capturing video ‘in all directions’, so it can be played back in a virtual reality headset like a video. The viewer with the VR headset can look in any direction, as if they are positioned where the camera is, so you must capture footage in every direction they might look. This is VR video at its simplest.
Yes, storytelling has a new vehicle and we couldn’t be more excited about the possibilities. We’re fully aware that our stories must support the medium. Because of this, we began asking ourselves “Why VR?” as part of our development process. Why is this story better told in this medium? What can we do to leverage the strengths of VR? Its been helping us stay true to our goal— to build truly immersive cinematic experiences that were never before possible.
Saschka Unseld’s the Story Studio group hopes to glean insight both artistic and procedural, not only on what triggers emotional responses in VR but even on how people watch a movie when they’re surrounded by it. They will then share their findings with other VR filmmakers to extend the cinematic landscape. They are working on an animation film Henry.
VR film making
VR Film making has a democratizing effect, turning film from its long-held status as a director’s medium, with well-curated shots, to something looser and more participatory
In the new world of VR cinema, filmmakers can create heat maps and conduct granular–or insidious, depending on your point of view–research on test groups of what’s working, potentially even down to every second and eye twitch.
A great medium to make films which convey emotions.Coming face to face with a giant but harmless animal? Wonderful. Gaining empathy for a far-off country hit by natural disaster? Sadness.
Watch Film Maker Chris Milk speak about VR 360
With New York Times, BBC and CNN taking leaps into 360 news , VR journalism has found its first steps. Just as young people in journalism school five years ago learned that Twitter was important to reporting, soon enough they might be learning how to film with a 360-degree camera.A lot of the questions about the importance of VR for journalism go back to empathy—the current buzzword in VR filmmaking. Taking a page from Roger Ebert’s assertion that a movie is an “empathy machine,” people excited about VR’s storytelling potential like to point out that nothing will make a person more empathetic to a protagonist than virtually living in their world. So when that protagonist is actually a resident of a war-torn country, say, or protester in the streets, that potential for empathy is quite sizable. And all of a sudden, 1.3 million people will have VR headsets in their hands—many of them for the first time—and the first thing they’ll watch will be the Times’ upcoming VR documentary, “The Displaced.”
VR psychological therapy
Virtual Reality (VR) Therapy is one of the emerging and most effective applications of VR technology, where patients are exposed to stimuli in fully controllable environments. Whether it is immersive, such as a CAVE-like environment (CAVE is a VR device that allows the user to be completely immersed), or non-immersive, such as desktop-like displays, the idea is to recreate a believable artificial environment that stimulates physical responses similar to those of a real environment that can be individually controlled, replicated, and tailored to the patient’s experiences. The patient is presented only with environmental features that he or she can control, such as difficulty level, complexity, and amount of stimuli. This enables a highly scalable and controllable environment.Researchers have also developed VR programs that are intended to distract patients and work as a virtual pain relief, e.g. for back pain and for children who are undergoing treatment for cancer.
A LAYPERSON’S GUIDE TO 360 VIDEO CAMERAS
The principle of capturing live action video ‘in all directions’ is easy to understand. Combine a number of cameras, arrange them in a circular, or better still… spherical layout, and film outwards in all directions simultaneously. Carefully arrange each camera (and lens) to overlap each other’s picture somewhat, and clever software can later stitch the video together.
Well that’s the theory… the truth is more complicated than that (although getting easier).
The first challenge is to capture the scene in 3D. There is currently a proliferation of 360 video cameras, that purport to offer VR video capture, however they only offer only 2D 360 video capture. Since all VR headsets are intrinsically 3D capable, it makes little sense to capture 2D video for VR, if future proofing is a concern. 3D is much more immersive.
In the past, the traditional approach to capturing 3D video was to use two cameras side-by-side, for left and right eyes. This is possible in a 360 video camera, you double the number of cameras, but this becomes expensive when the number of cameras required is already high. There are other limitations to this approach too.
A newer approach, thanks to software beginning to emerge, is to do away with the duplicate set of cameras, and use computational photography (a form of image interpolation), that reproduces viewpoints ‘in between’ the physical cameras’s viewpoints, effectively creating virtual left and right eyes.
Both approaches – left-and-right pairs of cameras, and computational photography – are equally viable for producing 3D 360 video, although the latter is likely to become the standard, as it offers several benefits.
An important consideration with 360 video is whether the camera is shooting single-axis horizontal video (traditional), or two-axis (vertical) as well. Think of your viewer wearing their VR headset… In traditional 360 video a user can turn their head from left to right, but what happens if they look up at the sky and then turn their head from left to right? What happens if they tilt their head 90 degrees to the side? A true VR camera will have cameras arranged in at least 2 axis of movement and ideally 3, to capture all the needed 3D visual data.
You can demonstrate the need for this by watching a typical 3D video and tilting your head 90 degrees to one side. You lose all 3D depth information because the original camera rig was shooting side-by-side video. It would need two more cameras arranged up-and-down to provide the necessary 3D image data.
Another way to demonstrate this is to hold up a thin book in front of your eyes horizontally. Adjust the book until you can see as little of its profile as possible. Now turn the book 90 degrees so it’s vertical. How much of it can you see?
It may look like a blurry mess, and this is perfectly normal, but you’ll see it appears wider. This is because your eyes are looking at the book from ‘either side’. You see more of it.
In the same way, a camera that only has two side-by-side cameras in a single axis cannot possibly capture the necessary visual information to reproduce the scene if you were to tilt your head 90 degrees.
An optimal 360 video camera will therefore have sufficient cameras, arranged appropriately, so that the world is captured from sufficient positions to cater for all possible orientations of a VR viewer’s head.