NVIDIA : Capture, Stitch, and Stream VR Content in Real-Time with VRWorks 360 Video SDK

To enable developers and content creators to capture and broadcast amazing 360 content, we're extending our VRWorks SDK to accelerate 360 video stitching.

There are two main types of content in the burgeoning VR market today - real-time rendered applications and captured 360 panoramic videos.

NVIDIA GPUs are well known for rendering real-time graphics at blistering frame rates for display on VR headsets.

While viewing 360 videos in VR doesn't require a beefy GPU, graphics processors are becoming increasingly important to the creation of these videos. That's because 360 videos are often created with up to 16-plus individual 4k cameras, then processed together into a single 360 panorama in a very computationally intensive operation called video stitching.

The nature of shooting 360-degree video - capturing and stitching together footage from myriad cameras and angles - creates many challenges in the editing process. Continuity between shots, visible seams and blurring on the edges, and mismatched color and exposure across cameras can result in an unrealistic viewing experience. The stitching and streaming process can take hours to complete, depending on the type of content and length of the footage.

VRWorks 360 Video will allow content creators to take video feeds from camera rigs of up to 32 cameras and stitch them into a single 360° video either offline or in real-time.* The SDK performs GPU-accelerated decode, equalization, calibration, stitching, and encode of the video with both high quality and high performance. In addition it allows for custom rig and camera parameters and various types of fisheye lenses.

The 360 Video SDK is also compatible with NVIDIA's GPUDirect for Video for low latency ingest of video from camera rigs into the GPU.

'Capturing and stitching 360 video is time consuming and computationally demanding.' says Masaki Miyanohara, CTO of STRIVR, 'NVIDIA's VRWorks 360 Video SDK will help accelerate STRIVR's workflows, delivering real-time, high quality 360 video.'

NVIDIA is showing the VRWorks 360 Video SDK in action this week at Siggraph in Anaheim, California. The demo features a notebook with a NVIDIA Pascal-based Quadro GPU using the SDK to stitch video from a 4k camera rig together in real-time with near imperceptible latency.

The SDK will be released publicly later this year for VR developers and content creators to take advantage of. Register on the VRWorks developer page to get notified as soon as the 360 Video SDK is available for download.

Keep up with everything happening at Siggraph at #SIGGRAPH2016.

*Number of video streams for real-time stitching may be limited by video ingest capability or system performance.