TVN TECH

Fox Flies All-IP For FIFA World Cup

Fox has created a modular 2110 system that ties into U.S. operations for its upcoming World Cup coverage. Both Fox and Telemundo, which has U.S. Spanish-language rights to the tournament, will see a series of firsts when the Cup launches in Qatar next week.

The 2022 FIFA World Cup in Doha, Qatar, which starts Sunday, will represent a number of firsts for the quadrennial soccer tournament. It will be the first time the World Cup is being held in the Arab world, the first time it will be played in late autumn and the first time it will be hosted by a single city since the inaugural event in 1930.

The tournament will also create a number of broadcast firsts for U.S. English-language rightsholder Fox and Spanish-language rightsholder Telemundo in terms of both their on-the-ground presence and volume of coverage.

For Fox and cable network FS1, it will be the first time that every announcer, reporter and analyst covering the World Cup will travel to the host country, with all matches called by in-stadium play-by-play announcers and match analysts and pre- and post-game shows coming from a massive set along the Doha waterfront. It also will be the first time the World Cup, whose 64 matches run through Dec. 18, falls in the heart of the NFL season. And it will be the first time that Fox deploys a new IP-based technology platform for remote production, which it will extend to other big events in 2023.

All-IP, Modular Platform

FIFA host broadcaster HBS (Host Broadcast Services) is producing all of the World Cup matches in 4K HDR, using a minimum of 42 cameras for each contest. Fox knew it needed a new infrastructure to handle the format and bandwidth requirements of ingesting those feeds, which it is augmenting with its own commentary, graphics and ENG camera crews, as well as to produce its pre-game and post-game shows.

So, the network, with help from systems integrator Diversified, built a format-agnostic, all-IP modular platform which it was able to fly into Qatar and then seamlessly connect back to its cloud-based broadcast centers in Los Angeles and Tempe, Ariz.

BRAND CONNECTIONS

“We are tied back into both our Los Angeles broadcast facility and Tempe broadcast facility with this system, with all of the various transmission paths that we need,” says Fox Sports VP of Field Operations and Engineering Kevin Callahan. “We’re not having to take away transmission path resources from other programming. What’s different about this World Cup, of course, is that it’s in the middle of football season. So, for us to be able to sit there and take away 60 contribution paths into the building during NFL season would be impossible.”

Fox’s Flypacks

The fully SMPTE 2110-compliant system actually consists of two redundant, identical “flypacks” specially designed to fold in half height-wise in order to fit in the cargo hold of a commercial airplane. One flypack is located at the World Cup’s International Broadcast Center (IBC) at the Qatar National Convention Centre and the other is at the Fox set six miles away on the “Corniche,” the road rimming Doha’s harbor. They connect to each other via redundant 100-gigabit-per second IP links, as well as to the Fox data centers in Los Angeles and Tempe and allow all locations to work as one networked system.

The flypacks are configured with the same TAG Video Systems monitoring, Riedel intercom and EVS ingest systems that are used back in the U.S., and they also tie into the media asset management (MAM) system that Fox Sports recently created with Google. After the World Cup concludes in December, they will fly back to the U.S. and be used as the core compound distribution system for the Super Bowl this February in Glendale, Ariz., working alongside conventional mobile production trucks. Later in 2023, the flypacks will then travel to Australia for coverage of the 2023 FIFA Women’s World Cup.

The 2110 Advantage

The concept of remote production infrastructure that can be shipped from event to event certainly isn’t new; NBC first created its container-sized RIBS (Rack In a Box System) platform for Olympic coverage back in 2000. What is new is that Fox is taking full advantage of equipment based on the 2110 standard to achieve a smaller form factor, more workflow flexibility and more powerful connectivity than ever possible with SDI equipment.

By designing its flypacks to fit on a pallet used for airplane cargo instead of within a shipping container, Fox can move its equipment across the world in days instead of weeks or months. Once at a venue, it can be configured rapidly because of the lightweight fiber connections it uses compared to the hundreds of coax cables that would have been used for SDI. That saved days of setup time in Doha this fall and potentially could save weeks in the future, Callahan says. And deploying the same systems that are used back in Fox’s U.S. plants creates a degree of familiarity that should make large remote productions easier.

“What it allows us to do is to be repeatable,” Callahan says. “Not that any of our events are cookie-cutter. But they all have various core requirements, and now we’ll be using the same system, the same gear over and over again, so that it becomes repeatable. It becomes something that is sustainable that we can do, and really make these large events part of our core infrastructure.”

Remote Data Centers

Peter Wharton, chief strategy & cloud officer for TAG Video Systems, says that creating such a modular system in SDI would be impossible just due to the sheer weight of the coax cabling, never mind the larger size of SDI gear. The two 2110-based flypacks in Doha represent a “massive system” in terms of technical capabilities, says Wharton, who notes that the TAG monitoring installation can handle up to 1,200 sources and drive 150 displays.

“These are basically remote data centers that are extensions of the main data centers that Fox has in Tempe and Pico, connected with these very fat pipes,” Wharton says. “Which means that they can do anything they want any time they want — there are no predefined circuits, there are no predefined paths, per se. You basically have bandwidth and connectivity available to you, so you really can on the fly do whatever you want.”

Belt And Suspenders

Fox’s World Cup technology plan is focused on covering every possible contingency to avoid any on-air problems. The new 2110 flypacks flew in separate planes to Doha, one on a cargo jet and the other commercial. Each flypack system consists of four cargo pallets, and each pallet has five racks (unfolded dimensions are 85 inches wide by 110 inches long by 96 inches high). And each system itself contains “two of everything,” Callahan says, meaning that Fox needs only half of one system, or 10 racks of equipment, to actually get on air. And Fox will still have a small SDI router in Doha for backup.

The flypacks handle signals using Arista IP switches under the control of EVS’ software-based Cerebrum routing system. Since HBS is producing at the 50 hertz (Hz) frame rate used in Qatar, Fox will rely on the Grass Valley Alchemist system to perform frame rate conversion to 60 Hz for U.S. distribution. Fox is also using an EVS ingest and playback system and Grass Valley production switcher. There are Grass Valley LDX 150 cameras at its stage location, while the ENG cameras deployed at the matches are a mix of Sony PXW-FX9s and VENICE 2s.

Fox’s Corniche set measures 148 feet by 33 feet within two stories, with more than 20 LED screens including an all-LED field. The studio shows will use new augmented reality (AR) graphics as presenter tools, doing some of the work that touchscreens have done in the past. The underlying technology will be Vizrt with Unreal Engine and Stype motion tracking. Fox originally planned to produce the studio shows in 4K HDR but didn’t have enough time to test the new AR graphics in 4K, so the studio shows will be produced in 1080p HDR and upconverted to 4K HDR.

Fox will distribute its World Cup coverage in 4K HDR through its Fox Sports streaming app and the various pay-TV partners who currently carry select 4K HDR sports coverage from Fox such as Big Noon Saturday college football. The Fox broadcast network and FS1 will downconvert the games to the 720p SDR format for mainstream distribution.

Some Remote Functions

With eight World Cup venues within a 21-mile radius, Fox will have announcers and a technical presence at every stadium in addition to its stage and IBC locations. But it is making use of its robust IP connectivity to the U.S. — two 100-gigabit and two 10-gigabit links — to do some things remotely. As it did for the 2018 World Cup in Russia, editors will work from Fox’s L.A. facility to put together packages for the pre-game and post-game shows. EVS replay operators will also be stateside, remotely controlling the hardware in Doha from Fox’s “Vault” facility in L.A.

Graphics operators will also work from L.A., controlling Vizrt “Viz Engines” that Fox has virtualized into the Google Cloud Platform (GCP) and using the NDI networking protocol to deliver the graphics to the production team in Doha.

“They’ll be sending NDI to us in a stage location here in Doha, and then we’ll be going NDI to 2110 to actually output the video,” Callahan says.

While Fox has tested NDI extensively and is currently using it some for some daily studio show workflows coming out of New York, the World Cup will be the first time Fox is employing it on a marquee event. Callahan says the technology is ready.

“In all the testing we were doing with it, nobody could pick out which was the locally generated Viz graphic and which was the Viz graphic that was coming via NDI,” he says. “It was pretty impressive.”

FIFA World Cup Tonight, the nightly wrap-up show that will air on both Fox and FS1, will also be produced as a complete “home-run production” with talent and producers in Doha but switching and compositing taking place in Los Angeles.

“Almost At The Speed Of Light”

Lumen is handling Fox’s long-haul connections, while local connectivity in Qatar is being provided by Vidi and Oordeoo. Fox will also have SRT and LiveU bonded cellular for backup links as well as a satellite uplink at its Corniche stage location.

Fox will send its feeds back to the U.S. with JPEG-XS compression, using the EVS Neuron Compress processing unit to achieve a data rate of around 330 Mbps including audio. The latency from video encode in Doha to video decode in L.A. will be about 124 milliseconds (ms), some 4 ms faster than it was in 2018.

“We’re at the point now where we can’t get any faster,” Callahan says. “We’re truly almost at the speed of light.”

One of the things that Fox likes about the IP-native TAG monitoring system is the ability to display both the native 2110 video and the JPEG-XS video it is creating with the EVS units on the same multiviewer.

“Because it’s on the same IP fabric, we’ll be able to see both compressed and uncompressed on the same display and do a comparison between them,” Callahan says.

Between 450 and 500 people in total will have worked on Fox’s FIFA production at some point, including stagebuilders who arrived in early September and left in October after completing their work. About 330 people will be on hand for the live broadcast production, with an international team full of World Cup veterans. Fox is also using local workers where possible, and has formed a partnership with Northwestern University and its satellite campus in Qatar to find production assistants and freelancers.

“They have a really good broadcast journalism school here, that is truly second to none, so they’ve been a good resource for personnel,” Callahan says.

“Football Is Better In Spanish”

That is the mantra of executives at NBC Universal, whose Telemundo broadcast network has the U.S. Spanish-language rights to the World Cup and will be broadcasting all 64 matches live. In addition, NBCUniversal’s Peacock streaming platform will be offering all of the matches live in Spanish. Like Fox, Telemundo will for the first time have talent and dedicated announce teams at each stadium for every match.

Telemundo is ready to deliver the “the best, most extensive coverage ever in the history of World Cups in the United States,” says Eli Velazquez, EVP, sports content for Telemundo Deportes. That means a total of more than 2,000 hours of coverage across Telemundo and Peacock and other NBCU properties including cable network Universo, TelemundoDeportes.com and Telemundo Deportes’ app.

Telemundo’s main presentation studio

Telemundo Deportes will have 10 distinct positions across Doha. There are four sets, all built by HD Studios and Black Walnut, along with various in-stadium positions including a studio in Lusail Stadium (the venue for the final) and a standup position at the IBC. The main presentation studio will be at the center of the Souq Waqif Marketplace; a second set will be along the Corniche; a third will be at waterfront Al Bidda Park, site of the FIFA fan festival; and a fourth will be in the interior of the Souq Waqif marketplace, which Velasquez says is a position exclusive to Telemundo.

Since the 2022 World Cup will represent the most compact event ever in terms of both the geographic footprint and the tight schedule of matches, Velasquez said there could be some “logistical challenges” in terms of just people movement between the venues.

“We’re going to learn a lot in the first week,” he says.

The first four days of the tournament will be streamed free without authentication required on both the Telemundo Deportes app and Peacock. After that time period ends on Nov. 24, viewers will have to authenticate with their “TV Anywhere” cable credentials on the app while Peacock will have a specially priced offer to sign up for its pay tier. For the entirety of the World Cup Peacock will also offer a free 24/7 dedicated World Cup channel with coverage, commentary, debate and replays, says Romina Rosado, EVP of Hispanic streaming for NBCUniversal Telemundo Enterprises.

Telemundo’s streaming coverage will be delivered in the 1080p HDR format, Velasquez says. “The quality of feed is excellent, and I think streamers will be very happy with the product they receive.”


Comments (0)

Leave a Reply