Broadcasters Begin Experiments With VR, 360

What if there could be more to watching television than viewing a high-res image on a flat panel — whether that screen is on the wall in a living room, part of a smartphone in the hand or a media tablet on a coffee shop table?

Certainly the consumer electronics industry thought it had the answer with all of hype around 3D stereographic viewing, which — despite the best efforts of avant-garde broadcasters and companies eager to sell new displays — ultimately fizzled.

Undaunted by that experience, however, some broadcasters like NBC Sports, Fox Sports, Turner Sports and CNN have chosen to push the envelope once more to bring virtual reality (VR) 360-degree productions to over-the-top viewers of special event content, like the Rio Olympics, the Kentucky Derby, presidential primary debates and political conventions.

“Wearing a virtual reality headset can really transport you into a scene,” says Jason Farkas, CNN’s VP of premium video and head of VR. “It’s almost like a time travel device that brings you to a place where you are not.”

And while consumer reluctance to wear 3D glasses is widely seen as contributing to the derailment of the 3D bandwagon, VR gives viewers the option to watch with a VR headset like an Oculus Rift or HTC Vive; use a smartphone or tablet in “magic window mode” that tracks the display of video content with the physical movement of the display; or simply swipe a finger side-to-side, top-to-bottom or diagonally to experience the 360-degree content.

“I think the headset experience is the richest experience, and that is what we want to produce for, but I also think there is such a broader audience for the magic window mode that live VR from that perspective can be great if you want to be part of a news moment as well,” he says.

In addition to its normal HD production of the CNN Facebook Democratic Debate Oct. 13, 2015, from the Wynn Hotel in Las Vegas, the cable news network streamed the event live to owners of Samsung GearVR headsets.

To do so, CNN worked with NextVR, a Laguna Beach, Calif.-based company specializing in live video production that combines virtual reality with stereographic 3D to immerse viewers in what is being captured.

For the production four NextVR cameras were placed strategically around the stage. With no zoom available in VR, it’s important to position the cameras as close to the subject as possible, Farkas says. “We actually went through some backflips to make that work.”

Specifically, CNN built into the set between where two of the debate moderators sat a small part of the set to house a NextVR camera. That special housing closely matched the appearance of the rest of the set.

“This got us really close to the candidates — about 15 feet away — but no one knew we had a camera right there on stage.”

A second NextVR camera was placed “off in the wings shooting onto Anderson Cooper’s moderator stand”; a third was “perched up and behind the candidates” to give the candidates’ perspective on the event; and the fourth was placed in the audience to provide “a sense of what it was like to be an audience member,” Farkas says.

Then proprietary signal processing from the NextVR stitched the video from each camera into a seamless stereoscopic 3D VR experience.

“All the people in the image you are watching in a headset have depth and volume, and that even further enhances the experience of being there,” says Farkas.

NBC Sports is another broadcaster to work with NextVR. This year, it produced the first live stereoscopic VR coverage of the 2016 Kentucky Derby that was streamed to users of Samsung GearVR headsets.

To do so, seven NextVR cameras were deployed. Five were placed trackside at Churchill Downs to capture the actual race. Two others were used in the paddock area for shots of the horses before and after the race.

Switching a dynamic event like a race required a different approach than a relatively static event like a primary debate.

To keep VR race viewers oriented, NBC Sports decided which cameras would be shown in the headset at specific times, says NBC Sports spokesman Dan Masonson.

VR viewers also could see “a Jumbotron in each camera frame” with the linear broadcast to provide further visual context. On the flipside of the forward-facing VR experience, NBC Sports inserted an odds board to round out the experience, he says.

“The Kentucky Derby is a unique experience — two-and-a-half hours of a show all leading up to two minutes of the race,” he says.

It was important to create an experience that allowed VR viewers to “feel like they were sitting at the finish line or in the paddock,” he says.

Alexander Lindsay, founder of Pixel Corps, located just north of San Francisco, took a different approach when producing 360-degree coverage of the Republican National Convention.

Equipped with a Nokia OZO camera, Lindsay captured live 360-degree coverage from three locations at the event — in the VIP section near the [Donald] Trump family, at the back of the Quicken Loans Arena in Cleveland and on stage.

“The VR experience really provides a new — and I would argue — more intimate view than what you would normally get even from attending the event,” Lindsay says.

[Lindsay declined to identify his client for the 360-degree convention production, citing restrictions imposed by a nondisclosure agreement. However, there is a link to a “360 LiveStream” from the RNC 2016 website. (http://bit.ly/29P8oQF)]

The OZO is a purpose-built camera with a unibody design that can be operated as a standalone to capture 360-degree video and audio, says Jyri Huopaniemi, Nokia’s head of formats and platforms.

Built into the mostly spherical OZO are eight 4K cameras, eight lenses and eight microphones. The camera can run standalone and record for 45 minutes to an hour or output a signal at 1.5Gb/s via an SDI cable, he says.

Users can record all eight cameras in 360-degree stereoscopic 3D that can be assembled in post or, with newly introduced software, stream 360-degree video live, Huopaniemi says.

For the OZO-based coverage of the Republican National Convention, all camera signals were sent via fiber to Nokia stitchers that assembled the video into a 360-degree experience before being fed into hardware encoders and streamed online, Lindsay says.

Both Lindsay and CNN’s Farkas agree these are early days for VR and 360, and that unlike the mostly failed rollout of stereoscopic TV, 360 and VR have a better chance of success. That’s because there is a way for early adopters to get personally invested in the technology with their own affordable 360-degree consumer cameras and VR headsets.

“I think it is important — whether you are starting with a Ricoh Theta S or you start using a Nokia OZO — to start experimenting with 360 now because it is something we see coming down the path very quickly,” Lindsay says.

Doing so is likely to reveal new opportunities for the technology in news, says Farkas. “Whether it’s a presidential debate or a war zone or a protestor or a natural disaster, there are plenty of stories where the environment is the story. And [in those instances,] I think 360 video and virtual reality will really pay dividends.”

To stay up to date on all things tech, follow Phil Kurz on TVNewsCheck’s Playout tech blog here. And follow him on Twitter: @TVplayout.