NEW YORK — The promise of IP networking and cloud platforms to broadcasters is that their essential functions can now run, and be run, from anywhere. But the physical location of people and assets still plays a major role in determining what new technology stations and networks invest in, according to senior technology executives who gathered Tuesday at TVNewsCheck’s 2019 NewsTECHForum here.
Moving to smaller physical footprints — or moving hardware completely from expensive urban environments to less expensive real estate — is a big driver in the transition to IP networking. The ongoing virtualization of news systems also allows many key functions to be hosted in remote data centers or public cloud platforms, said executives speaking on the panel “Top Technologists on Tomorrow’s Challenges,” moderated by this reporter.
But at the same time, some of the country’s biggest news operations still want to hold high-value content close to ensure quick access for breaking news coverage — and that can mean sticking with on-premise storage for much of their near-line and archive content.
“Real estate plays a very big role,” said Jeff Birch, VP of engineering for CBS Television Stations. “We’re in some of the biggest markets in the country, so absolutely the ability to reduce your footprint, the ability to reduce your carbon footprint by minimizing the amount of hardware you have on the ground, is always a consideration. But it’s a balancing act.”
Birch said the decision of what to keep on-premise and what to move to a data center or the cloud varies significantly between individual stations, even among big markets like New York, Los Angeles and Miami, and so each station has to deal with it on a standalone basis.
“Certain material, you can put up in the cloud, you don’t worry about it, you don’t think about it, because it’s storage,” Birch said. “Other material, you may need five minutes from now. I have a little bit more of a provincial attitude — I don’t want to give up control of our assets. I’m putting my destiny in the hands of somebody I don’t know, I don’t see, and when the news director calls and says why can’t I get my hands on that story, or so- and-so just passed away five minutes ago and we need it, well, we better be able to get it.”
A Gradual Transition
Overall, the CBS stations are easing into the IP transition. Perhaps 20% of the group’s overall infrastructure is IP at this point, said Birch, who noted CBS has so far been placing IP technology in “islands” where it makes operational and financial sense, such as master control routing. The next step will be studio control rooms, IP camera and IP production switchers.
“We’re going to be pushed into IP by the manufacturers, because just like they stopped building analog equipment and we had to all migrate to digital, they’re going to stop making HD-SDI and we’re going to go to IP whether we like it or not,” he said.
Meredith Local Media Group is also taking a gradual approach, said VP of Engineering and Technology Tom Casey, and is just looking at installing some IP routing cores at a few stations now.
“We’ve had a couple of cores come up over the last couple years, and it didn’t make sense for us at that point,” Casey said. “We actually do have a couple coming up, and it’s definitely a different conversation now.”
“The guys out at the stations when they look at it, they think of a core traditionally as just the router and the distribution and the patching,” Casey added. “There’s actually a lot more than that. You have to look at the master sync generator system.
“We have HD-SDI video feeding our audio consoles for news production,” he said. “So there’s a place where you’ve got to walk things back and take a look at what else is involved, beyond just core routing and switching and cameras.”
Meredith has already virtualized its news playout and editing functions, while its transcoding happens in the cloud. Most of the systems that have been virtualized still reside at the stations or in Meredith’s own data centers.
“Not much of that has moved away, and a lot of that is the timeframe that the content is actually needed when it’s requested,” Casey said.
ABC was an early adopter in IP networking and has IP routers for both news and master control at its New York headquarters. The master control router uses the older SMPTE 2022-6 IP switching standard, which it plans to stick with for the foreseeable future. The news router is fully 2110-compliant though it doesn’t yet deal with many 2110 sources, but instead uses gateways to interface with existing HD-SDI gear.
Marcy Lefkovitz, VP of technology and workflow strategy for Disney ABC TV, says the news division was looking ahead in moving to 2110, which allows for greater density of video and audio sources.
“We know that that’s where we want to be,” Lefkovitz said. “And we know that certainly on a news production level that that is the right format to be on.”
No Single Solution For All
Mike Palmer, CTO for storage vendor Masstech Innovations, said that his company’s broadcast customers are looking at a wide range of storage solutions, particularly when it comes to archiving news content.
“If you look at a small station in Iowa, they have different needs than a large station or network in New York,” Palmer said. “They can make different decisions, they value their content differently, their content is used differently.
Palmer noted that doesn’t mean that everyone needs to go to the same solution. “Cloud is great for some workflows, and it’s really awful for others,” he said. “On-prem is great for some workflows, and it’s really awful for some others. So you really need to do a deep dive into that, and that’s what we’ve been spending a lot of time doing in the last year and a half.”
In helping customers make storage decisions, Masstech has developed its own modeling tool called Foresight that stations can use to predict what their long-term storage costs will be across a number of different scenarios and vendors, for both on-prem and cloud storage. Palmer said that one mistake stations make when evaluating the cloud is not making a distinction between different content types and the impact that has on egress costs.
“There’s a lot of talk about moving to cloud for playout and long-form production, where you’re dealing with large files, and the files are generally of a known value,” Palmer said. “You can begin to do some calculations about how much it will cost for me to put this up there, how much it will cost me to get it out, versus how much value it will have at that point in time.”
Palmer noted that one of the big concerns is egress at that point, because the files are so large.
“But that’s in entertainment and playout,” he said, noting that news is a different environment. There, particularly with archived content, most may never see the light of day after the first month, or three-month period.
“But we don’t know what part of that content we’re going to need back, so we have to store it all,” Palmer said. “The duration of that content is also very short … and we’ll pull relatively little of that back if we’re talking about a local station.”
He noted that for local stations, the egress charge may be practically nothing as compared to their storage charge over time. “So that’s not even a concern at that point,” he said.
Palmer cautioned that broadcasters have to be careful about the buzz to which they’re listening.
“The egress charges are important to some customers. We have to make sure that you’re one of those customers that it’s going to be relevant to before you buy into that, and overly weigh that in your decision about what you do,” he said.
ABC can relate to that point. While its cable networks already play out from the private cloud and the broadcast network is moving its archive workflow for VOD entertainment content to the cloud, ABC News hasn’t used the cloud much to date.
“We’re still using probably the exact same cloud calculation Excel spreadsheet that we pull out every year, and we jam in our numbers of how much does it cost, how much ingress and egress do we expect on this workflow, and what are the time constraints of the workflow that we want to migrate?” Lefkovitz said.
“Is it highly transactional, is it something that we can put a little bit farther away, and we can wait for the access?” she continued. “And we see what the cost benefit is. Every year it gets a little bit better, but it hasn’t made sense for many of our workflows.”
In fact, ABC actually just purchased a new LTO (linear tape optical) storage robot to house its high-resolution news archive. The old-school LTO technology has allowed for one big change, however, which is to move the archive out of expensive on-premise real estate on the Upper West Side of New York.
The primary LTO is now situated in a data center in the New York suburbs, close enough to remove any network latency concerns, and a smaller backup LTO system, along with a duplicate set of tapes, is located at ESPN headquarters in Bristol, Conn., with similar high-speed connectivity.
“We thought there was risk with putting the high-res workflows in the cloud in that the users might not be able to retrieve a piece of content as quickly as they needed to,” Lefkovitz said. “Somebody dies at 6:25, five minutes before World News [Tonight], you hadn’t touched that person’s content in 10 years, it’s way deep in glacier storage, but we need it immediately.
“We like to have a little bit more control over the retrieval queue for archive,” she said.
While Lefkovitz is comfortable with the security measures offered by the cloud today, she and the other panelists said that security remains a big concern as broadcasters adopt IP technology.
New Risks
The fact that so much broadcast technology today is connected to IP networks exposes stations to risks they never had in the IP world, said Birch, who referred to the recent hacking of a Washington station [KHQ Spokane] that took down their automated newscasts for several days. He says that CBS staffers spend as much time talking about security these days as they do discussing the latest big story.
“It’s an everyday conversation, and it causes a lot of internal strife and conflict because we want to be able to do something as news people — go get raw video, go get stories, get whatever we need — but a lot of times that runs right into the face of the IT security rules which are necessary,” Birch said.
That has made for a conflict, he said.
“Every day it’s a conversation about how do you beef up the security so that you’re not vulnerable, but it doesn’t stop you from doing your business,” Birch said. “There’s no easy answer to it.”
Watch the full video of the session here.
Comments (0)