Thanks to native support on Facebook and YouTube, virtual reality and immersive 360? video have reached a tipping point in which production, distribution, and consumption are now available to almost everyone. From enabling new forms of empathy to exploring social issues in an on/off-line world, we are more empowered than ever to experiment with new ways to convey stories and ideas. There’s one problem though: Many creators are still struggling to find the best way to tell stories with this medium.
In this post, I’ll be going into a few of the storytelling and technical challenges of 360? video, and share some production methods I’ve been prototyping for this new medium. Even better, you won’t need a 360? camera to try it for yourself.
Tools Enable, Storytellers Enthrall
Early filmmakers in the 20th century had a new tool in the motion picture camera, but had yet to fully explore how to use it to maximum effect. The closest frame of reference for film was theater, so in the silent era, many directors placed the camera “fifth-row center,” and filmed movie scenes as if they were sitting in a theatrical audience—no cuts, close-ups, zooms, or pans.
Over time, filmmakers experimented with and ultimately developed an established visual language for the new medium. Think of it as the film equivalent of grammar—sentences, paragraphs, and so on.
Many of the 360? immersive videos we’re seeing today are effectively in the same place as traditional film was 120 years ago. Since the camera can see in all directions —and the user can look in any direction at any time—the classical visual language of film no longer applies. Creators are having to experiment all over again to develop a new visual language that works in this medium.
360? video is great for first-person-like views of extreme sports, but it poses a number of challenges for producing narrative content. If the camera can see in all directions, what do we do with the background? Is there even a background anymore?
In response to these challenges, the creators of some first-wave 360? videos have chosen to plunk the camera down and orchestrate a performance around it. In other words, if the camera can look everywhere, let’s put the action everywhere. Sound familiar?
It’s one way to deal with the format, but I’ve been exploring other ways we might begin to tackle the storytelling challenges of 360? video, and also address some of the production hurdles we encounter when we shoot in 360?.
One of the first instincts creators might have is to buy a 360? camera. But that’s a lot of money you really don’t need to spend, and I’d even argue that in some cases, a 360? camera may actually limit creative possibilities. Instead, we can repurpose our existing tools to give people the power to create immersive content without requiring additional gear or studio considerations.
A Partial View of the World
To address this challenge, I’ve created techniques that use traditionally-shot 2D film footage to create composite, immersive 360? video worlds. These GIFs showcase a quick overview of the technique, in which four traditional shots are blended into a single 360? scene.
So what exactly is happening here?
To start, let’s first look at what a spherical 360? scene looks like when it’s unwrapped:
This is called an equirectangular grid, and it is one of the standards that Facebook and YouTube have aligned around. As you can see, there is significant distortion at the top and bottom of the image. These are the the poles that, when spherized, represent the top and bottom of the view. The centerline, however, remains relatively distortion-free, and luckily for us, that’s the horizon line in all directions.
When wrapped, this distorted grid becomes a clear shape.
Creating Your First 360? Scene
This workflow uses Adobe Premiere Pro, but if you prefer another video editing suite without native VR export, don’t worry, there’s a workaround at the end.
First, let’s get the tech specs out of the way. You’ll want to setup your project and sequence with these settings:
- For best results, I recommend editing in a 4K workspace (3840x2160px) at either 30p or 60p.
- When compositing 2D shots into the scene, I recommend sticking to the equator of the image as it will have the most predictable results.
Once you have that workspace, import your desired 2D footage. In my example above, I used footage composited on a black background that blends seamlessly into the (also black) background of the 360? world, resulting in invisible edges around each of the shots. I was also able to luma-key (effectively a form of green screening) the background so I could overlay several shots to create large scenes composed of multiple individual shots. In my real world shoots this technique allowed me to feature multiple individual setups, helping a tiny set to punch well above its weight.
The Secret Sauce: 360 Metadata
Now you’ve got your 360 scene prepared and are ready to export it for upload. Here’s where the magic happens: To get YouTube or Facebook to see your 360? video as such, you must include the proper metadata.
This is much easier than it sounds. If you are using Adobe Premiere Pro CC (2015.3) this feature is built in. Find it in the export settings at the bottom of the “Video” tab here:
- Export your 4K video to H.264 as you normally would.
- Inject the required metadata using the YouTube Spatial Media Metadata Injector. This is not as scary as it sounds, and it will enable both YouTube and Facebook to detect your video as 360? content.
— First, download the YouTube Injector tool for Mac or Windows.
— Unzip the file, then use the application to open up your video export.
— Select “My video is spherical (360)”.
— Process your file.
That’s it. Just a simple press of a button, and your video will be recognized as 360? content on all the major platforms.