As broadcasters have relied on remote production workflows over the past nine months to maintain news coverage through the COVID-19 pandemic, a challenge cited by many is getting return video to talent in the field, particularly anchors or meteorologists working from ad-hoc home studios.
Using bonded cellular or public internet connectivity to contribute feeds to is nothing new. But providing the confidence monitoring that talent is accustomed to in the studio back over an IP connection is far more difficult. That’s because the compression schemes used in internet transport generally add significant latency to the signal, with a delay of several seconds or more by the time the video being generated by the control room is viewable by the talent.
However, several major U.S. networks and local stations have found a solution to the problem in a small Japanese firm, Techno Mathematical Co. (TMC), and its low-latency encoding and decoding technology. TMC’s encoders are designed specifically for return-feed applications and achieve a latency as low as 100 milliseconds, or a tenth of second, according to the company. And the company’s decoders are available as either compact hardware units or as software decoders that run on Apple iPhone, iPad or Apple TV products.
Wide Adoption
After initially being adopted by Japanese broadcaster NHK, TMC’s technology is now being used by NBC, MSNBC, CNBC, Fox News Channel, ESPN and WCBS New York in a variety of return-feed applications.
NBC uses the TMC decoders to provide teleprompters in the field via the iPad app, as well as real-time video and graphics. And since March, WCBS has used TMC to deliver return video to meteorologist Lonnie Quinn as he presents his weather forecasts from an improvised chroma key setup in a barn at his Connecticut home.
“It provides a really good home base-to-field application, and there are a bunch of things that make it quick and easy,” says Jeff Coneys, VP of operations and engineering for NBC News’ Western Region, of TMC’s technology. “But the super low latency is what it makes stand out.”
While TMC encoders have been used by U.S. customers like NBC for several years, business has taken off due to COVID-19, says Tim Ogawa, TMC’s VP of international sales. Existing customers expanded their use of the software decoders with the majority of talent working remotely from home studios.
“We’ve had a very busy five or six months,” Ogawa says.
Proprietary Algorithm
TMC was founded 22 years ago by University of Tokyo professor Dr. Masafumi Tanaka and since then has developed compression technology for a variety of consumer and professional applications. The company’s key technology is a proprietary compression algorithm, DMNA (Digital Media New Algorithm), which is designed to deliver fast image processing with low-power consumption and a low CPU load.
The secret of DMNA compared to other compression schemes like H.264 and H.265 is that it is focused on return feed applications, says Yasuo Suzuki, GM of TMC’s product development division. As such, it emphasizes low latency and a low data rate over absolute picture quality, since the end customer isn’t supposed to be a viewer at home but instead a professional in the field.
“We focused on latency most carefully,” Suzuki says.
While H.264 might show a lot of artifacts when set at a bit rate of 1 Mbps, Suzuki says, DMNA delivers a usable, clear picture at that rate. Meanwhile, H.264 or H.265 might deliver a better picture at 10 Mbps. But regardless of the bitrate used, DMNA’s latency stays the same.
While TMC initially started as a software-based business, it began to make hardware encoders about seven years ago for NHK, which was looking for a low-latency IP-based return feed system for its satellite newsgathering (SNG) trucks. Since NHK’s satellite feeds already had a latency of 350 milliseconds, the broadcaster was seeking a latency of only 100 milliseconds for the IP return system to support confidence monitoring and teleprompter applications for daily newsgathering. TMC was able to meet the 100-millisecond requirement, Ogawa says, and NHK subsequently bought the company’s compact encoder and decoder for all 54 of its stations in Japan as well as its SNG trucks.
Current Products
Today, TMC makes different versions of its compact IP encoder with HD-SDI or HDMI inputs, which can deliver encrypted transmission of video and audio over the internet at bit rates ranging from 128 kilobits per second (Kbps) to 3 megabits per second (Mbps), as well as hardware decoders that will output either HDI-SDI or HDMI. A maximum of 20 decoders can be connected to one encoder. The encoders cost around $6,000, while the decoders run about $4,500.
The company also makes software decoders for the iPhone, iPad and the Apple TV streaming device. While the decoder apps can be downloaded through Apple’s App Store, using them requires the purchase of a $1,200 software license, which runs for two years with a $150 yearly maintenance fee kicking in after that.
At the request of U.S. broadcast customers, TMC has settled on a pool licensing model where instead of dedicating a license to an individual user or device, a number of licenses can float among registered personnel. If a broadcaster purchases 20 licenses, that is the number of concurrent users it can have at one time. But hundreds of personnel in total could download the decoder app to their device (for example, Ogawa says NBC personnel have downloaded 800 decoder apps).
NBC began using TMC’s encoders and decoders about five years ago, when it started to set up home studios for anchors and frequent on-air contributors like former FBI agent Clint Van Zandt to more quickly respond to breaking news. The compact hardware decoders were a simple and effective way to get teleprompter and real-time return video into anchors’ and contributor’s homes via the internet.
What dramatically expanded NBC’s use of TMC was when the company created software decoders for iPhones and iPads, and successfully negotiated to distribute them through the App Store.
Use Cases For NBC
“That was really a game changer for us,” Coneys says. “We already had a hardware solution for low-latency video and teleprompter. Now with the app-based thing that really blew it up, and we could really use it in the field now. You can have a correspondent sitting outside a courthouse, waiting for a verdict and watching on their phone a feed of whatever they want in the field. They can have it right in front of them and have it dynamically, without going to a third party or having a producer in their ear.”
For storm coverage, NBC uses the TMC system to feed real-time graphics and video to producers in the field. And it also makes heavy use of the TMC app for teleprompter on the iPad, using two-way mirror (also known as beamsplitting glass) systems, which mount over the lens of a field camera. The iPad lies flat on a horizontal platform below the glass and reflects the inverted text scrolling on its screen so a reporter can easily read it, while the camera shoots through the glass and captures a normal image.
After successful use at the network, TMC started to be adopted across NBC’s’ various news operations, Coneys says, as well as stations and sports divisions. NBC now has 19 TMC encode channels out of New York, six out of Los Angeles, four out of Washington, two out of London, 12 out of CNBC in New Jersey and two out of NBC News Channel in Charlotte, N.C. Coneys estimates there are roughly 100 concurrent decoder licenses across the NBC news properties.
Coney isn’t sure exactly what latency TMC delivers for NBC, just that it’s a very small delay. “In our experience there’s no need to measure it,” he says. “If you’re less than a second, that’s pretty inconsequential — the prompter is where it needs to be.”
CBS’s Use
WCBS New York began using the TMC technology several years ago to support meteorologists working from its “Mobile Weather Lab” SUV, installing the app on 12-inch iPads that allowed them to see return video. Several reporters also used the decoder app on their iPhones, and before COVID-19 the station had purchased about 15 licenses. WCBS has found the latency of the TMC encoder to be around 130 milliseconds over an LTE connection, with effectively cuts the round-trip delay to a reporter in the field in half.
“If you have an iPad in the field with LTE service, you can see the video from the control room with minimal delay,” says Rich Paleski, WCBS director of broadcast operations and engineering.
Since staffers evacuated the station in mid-March, WCBS has expanded use of TMC’s technology. Some talent have connected Apple TV boxes with the app to their home TVs, while others like Quinn rely on the hardware decoder with HDMI output fed into a dedicated monitor.
While a delay of three or four seconds through a bonded cellular connection might be workable for a reporter answering questions from an anchor in the field, that isn’t feasible for a meteorologist standing in front of a green screen and looking at a monitor to see the output of the station’s weather graphics system in order to get his or her next cue. However, WCBS has found that it can reduce the latency of the contribution feed to the station significantly with some additional measures.
WCBS leases dedicated internet connections through CenturyLink at the station, which reduce the latency of incoming feeds from bonded cellular systems like LiveU. When using the encoder of a bonded system in conjunction with a home broadband connection, that reduces the latency of the incoming video from the home studio to around one second, which is mostly due to compression.
Additional frames of latency are added in the control room by frame-synching and switching before the return feed is sent back out. By using the TMC encoder, WCBS is able to keep the end-to-end delay to 1.5 seconds or less, which allows Quinn to accurately present the weather map each night.
“By the time his video leaves the control room, you’ve got an honest second of delay, and you’re not going to remove that,” Paleski says. “That’s why it’s important to remove as much of the delay as possible on the return channel going back. When Lonnie first tested this, he said if you can keep it within a second or a second and a half, that’s livable.”
New Products And Competition
TMC has expanded its product portfolio, with a new hardware control unit for its encoder that provides tally and intercom functions at the station and works with software decoders in the field to facilitate better communication between directors and talent (Ogawa says that product was initially suggested by Paleski). The company has also come up with a quad-split decoder for the iPad that allows for easy real-time remote monitoring, which it expects will be a long-term requirement for many stations.
TMC in not without competition in the low-latency encoding space. Haivision’s technology comes close, Coneys says, and NBC relied on Fujitsu for years before it started using TMC. He says the real differentiator for TMC is its ability to work on the iPhone and iPad through its decoding apps.
“The beauty of it is that anything on our router can very easily be sent out to anybody in the field as long as they have a cell or Wi-Fi signal,” he says. “That’s very helpful and very powerful.”
Comments (0)