Virtual Production to VR with CBS Digital: a case study

Over a hundred of 8K textures have been used in this Daredevil VR experience thanks to Granite for Unreal texture streaming system.

We’ve been working on game engines for many years now and it’s been amazing to see how video game technology and real-time render engines have found their way into completely new industries.

This is why we listened with the highest interest CBS Digital's Craig Weiss and Jim Berndt present their work on stage at SIGGRAPH 2017. After working with them on their virtual production tools for some time, it’s rewarding to see the results being showcased to an audience of highly qualified technicians in this world-leading conference.

You can watch the full SIGGRAPH 2017 presentation by Craig and Jim from CBS Digital in the Autodesk Vision series below.

The general idea behind virtual production is that you visualize an approximation of the CG directly on set, using the live captured animation or video. This way, the director immediately sees how the live performance is going to look with the CG characters or composited with the background CG, and can make quick calls on what works and what doesn't. CBS decided to use Unreal Engine 4 for the real-time visualization and we've worked with them to customize the game engine to fit their workflow. One major advantage is that these previsualization assets can be used in the VR experience as well.

CBS Digital is specialized in broadcast visual effects. And we all know that broadcast is specifically challenging because of the time and budget limitations. So CBS Digital has become experts in using virtual sets to remove the need to have elaborate sets or going on-site. They can now use a set with only a minimal number of props (only a bed for example) and mainly use a virtual set to fill in the scene. This way, even a hotel room can be used to shoot. This creates more opportunities to fit in the schedule of busy actors.

Netflix contacted CBS based on their virtual set experience to build a Daredevil VR experience that would immerse the fans into their favorite show. The great thing about having these virtual sets is that you already have the 3D assets to build your environments. However, assets for special effects have completely different requirements than for video games or VR experiences. CBS had to figure out how to make the content run on an average computer. If you’re familiar with Graphine, then you know this is the challenge we’ve been focused on for many years now.

This is how we've helped them. In their presentation, CBS mentions that they wanted to use their UDIM textures, a format typically used by VFX companies, with the Unreal Engine. Thanks to our Granite for Unreal plugin, they were able to automatically import the content without any manual work or quality degradation. It's exciting to see how they managed to use over a hundred 8K textures in the engine and keep the photorealism of the scanned assets.

“Texture wise, because you cannot predict where a person is going to be at any given time, they can get close to a wall, can get close to a poster. With [Granite] texture streaming, you get the ability to read the fine print on a poster which is really amazing. Really cool. You are unconstructed when it comes to textures. You can have a 256K texture if you want to.”
CBS Digital - Jim Berndt (min 32:20, video below).

Virtual Production to VR with CBS Digital from Autodesk Media and Entertainment on Vimeo.

Leave your information