Cloud-connected cameras used to be the domain of security systems, but the benefits for broadcasters could be huge – even if implementing them remains challenging. David Fox investigates.

cloud connected

Bandwidth boost: Smartphone acquisition using a Sony smartphone over 5G

Thanks to Covid-19 much production has moved to the cloud, from acquisition to delivery. Previously the only “cloud-connected cameras” were security cameras, but now broadcasters are connecting almost anything to the cloud, even if it isn’t always easy.

“The internet was not designed to deliver video,” said Raymond Thompson, director of market solutions - broadcast and media, Avid, so users must choose one of many IP protocols, all involving compromise (quality or latency). “All the major manufacturers have enabled IP contribution directly from the camera’s user interface giving users options for how they want to do contribution (proprietary protocol: Zixi; open source protocol: SRT; industry standard: RTMP, etc). Each has its own benefits in terms of the techniques employed to deliver video reliably either into a cloud or on premises environment.”

Avid did a tech preview at IBC2019 with Haivision and Microsoft showing IP contribution (from a Makito encoder and a mobile device with SRT enabled) into an Avid production environment. It used an agnostic software-only system (FastServe | Stream – coming soon) to transcode and rewrap the incoming IP stream in the cloud and delivered to an Avid production environment for fast turnaround workflows.

Anything the big manufacturers introduce “needs to be compatible with today’s SDI workflow, because if we were to make a big bang of introducing only IP equipment it would be too big of a jump for many customers,” said Ronny van Geel, director of product development, Grass Valley.

Grass Valley’s new LDX100 camera (shipping next month) is completely IP native, using open standards. “There are so many dialects with IP that you need to make sure everything works in one total ecosystem.” It uses field-replaceable SFP modules for connectivity, to work with all types of infrastructure.

“We cannot expect customers to go in one big bang to full IP because it means everything from synchronisation, audio, the teleprompter, everything needs to be IP. And for many people, that will be a very big investment,” he added.

JVC

Secure Reliable Transport: JVC’s GY-HC500 Connected Cam UHD camcorder

One camera company actively pushing cloud connectivity is JVC. Its Connected Cam range supports SRT, Zixi and RMPTS streaming and can be remote controlled via a web interface or IP-based broadcast remote control panel. So, “in a broadcast environment, it is fully possible to have normal engineering control of the camera images,” said John Kelly, general manager, EMEA Professional Business Solutions, JVC. The cameras also do return video and IFB audio over IP, opening up a range of cloud-based and remote-production possibilities.

Sony has several cameras offering wireless connectivity, and its XDCAM Air cloud service allows them to connect securely. This “means they can be managed as a suite of cameras rather than individually,” said David Hedley, business head, content management and distribution, Sony Professional Solutions Europe. XDCAM Air allows interviews to be uploaded as soon as they’ve been completed, while pictures and quotes can be published online immediately.

Spring into action
Richard Payne, technical development manager, Holdan, added that Panasonic has had P2 newsgathering cameras for some time “that send low-quality rushes for editing and then high quality later, via FTP.” They’ve been used by Associated Press and others, and mean that only the shots required by the edit decision list need be transmitted at higher bandwidth. “It is very clever. Everything is timecode synched and married up.”

One problem with much IP output is that its quality is lower than broadcasters would like. Most IP streams are 4:2:0 8-bit, delivering no more than 20Mbps. “This is good enough for control and proxy editing. If you trigger recording as well [to edit the high-quality video later], you need the proxy to have the same timecode as the master footage, but not all systems can do this,” he said – although this could be overcome using external recorders.

SRT (Secure Reliable Transport) is fairly new, but already important “because it is low latency and good at going over the dirty internet,” said Payne, but he recommends using NewTek’s NDI system, as it allows use of both compressed and uncompressed video at a higher bitrate, at low latency and with timecode.

Such a system has been used for the latest iteration of the BBC’s Springwatch nature programme, alongside inputs using SRT and RMPTS, marshalled by a Sienna ND system from media production specialist Gallery doing “vision mixing, routing, monitoring, and outputting in whatever format was needed,” said Mark Gilbert, Gallery’s chief technology officer. The show used “a completely virtualised production control room and facility infrastructure that used no hardware but the computer it runs on.”

Behind the scenes filming

Springwatch: Used a completely virtualised production control room

Springwatch mainly used traditional broadcast cameras, fed into Kiloview boxes (SRT and NDI encoders) to allow them uplink baseband to the cloud. This allowed them to produce four shows a week on linear TV, plus three shows per day for online. One innovation was offering viewers a quad-split multiviewer “so they could watch four things at once, just as the production did,” as 24-hour live streams. All production was in the cloud, with everyone working from home, and the system output 20 to 30 separate feeds for monitoring or delivery.

One key factor in cloud production for Gilbert is monitoring. “It is critical to have super-low latency monitoring.” This was done using SiennaLink, which is optimised for low latency and delivers video fast even if it has drop a frame, whereas something like SRT is optimised for broadcast viewing so will try to deliver video with no loss, which leads to latency.

“In the old mindsets, everything would go back to one central equipment room. So even if you have a remote production with screens locally, you would send everything back to home base and then back to the field,” said van Geel. “Now with a camera that is completely IP enabled, it means that the content for the local screens at a venue can just be subscriptions to the camera.” Whoever needs its output can just use it. If a camera is in the UK and the editor in Japan, there’s delay, but local screens don’t suffer from it. “Of course, you have to make sure that you’re not too far back in time that the intercom doesn’t work anymore - so you’re behind the action.” But he’s found that up to 100 milliseconds delay, even slightly more, is doable.

Delay can also be an issue with remote-controlled cameras, particularly robotics, but van Geel feels AI (at the location) could be the answer here, or a move to shooting more static scenes and taking a cut out of the image, rather than physically moving the camera head.

grass valley

Overcoming latency: Grass Valley’s van Geel would like to see IP implemented without delay

With IP, the impact on latency can be less than with SDI, where every product in a sequence, even if it introduces a single pixel delay, relocks everything to the frame sync. “If you have three devices in series, and they’re all delayed, one pixel, you have three frames impacted because they all resynchronise.” With IP, so long as the timestamp stays with the content, two half-frame delays in series only introduces one frame at the end rather than two. That “compensates partly for all the delay impacts that many people see, so the potential of it is huge. You need native IP stuff, because internally if it still is an SDI device with IP as the surroundings, you still pay the price,” said van Geel.

Delivering the news
Several Sony customers use cloud technology to upload news clips and also stream directly from camcorders such as the FX9 straight into the studio. “During the current pandemic, particularly in the hard news arena, consumer products and networks have been used successfully, although sometimes obvious to the viewer, to enable news organizations to get accurate, fact-based information to viewers, fast,” said Hedley.

Some productions are “delivering IP kits that include IP-enabled cameras so talent simply has to connect the device to a local internet connection and the cameras are already pointing at the proper IP address to make productions from home possible,” said Thompson. “Zoom is also taking centre stage in many productions and other similar solutions that are enabling groups of people to easily connect in a production scenario as if they were sitting on set together. People are leveraging Zoom, Teams, Zixi, SRT and standard protocols like RTMP for delivery from cameras, encoders, and mobile devices.”

“In this current climate of not being allowed any guests in the studios, we’re doing a real wide mix of down-the-lines,” said Tim Guilder, ITV’s technology manager. “We do lots of Skype, Zoom, Facebook portal calls and LiveU. Some guests have individual preferences on what they’d like to use and some guests or presenters have presented from home using kits that we have sent out. So, whilst we’re not using ‘cloud cameras’ as such, we are using everything from webcams and smartphones to remotely-controllable PTZ cameras like the Panasonic AW-UE150 and the occasional SNG camera (at SAGE social distance of course) to keep our shows content rich. Some of these practices may stick, but we will also welcome the return to having some guests in the studio.”

One of Guilder’s favourites was “a great interview with travel expert Simon Calder, live on Good Morning Britain via Skype, on his smartphone with Apple Airpods. First, he was in the departure lounge and then we came back and he was on board one of the first easyJet flights since lockdown. He was talking to Piers Morgan and the easyJet hostess was asking him to put his phone away as they were preparing for take-off. Was it technically perfect from a sound and vision perspective? No. Was it both newsworthy and entertaining to watch? Absolutely.”

Covid after effects
Kelly sees many use cases for cloud-connected cameras, particularly in light of how Covid-19 has impacted production. “We’re seeing an increase in customers requiring to shoot interviews remotely without the physical presence of camera operators. This can be achieved using IP remote control of the cameras to fully control the shots, with the possibility to FTP high quality online files directly for edit. We have also seen a particular increase in demand for music/concert applications where live streaming and IP-based remote camera control is required,” he explained. By adding JVC’s Connected Cam Studio, a cost-effective IP vision mixer, users can create an all-IP system for multi-camera acquisition and delivery.

For Gilbert, the pandemic hasn’t just forced even conservative broadcasters to move to the cloud more quickly than they’d like, it has also allowed them take risks, because they have a global crisis to blame if anything goes wrong, which has meant “we’ve seen a huge amount of innovation going on in terms of workflows.”

Connecting cameras to the cloud can mean any camera with an IP connection is usable. In the US, Gallery worked on the NFL Draft, when college footballers are picked for pro teams, with everything being done remotely for the first time. It was responsible for 64 camera feeds of all the coaches and general managers, which used iPhones on tripods with ring lights, using SRT streams into an Amazon Web Services cloud via Sienna. “The cameras looked fantastic,” he said.

Looking ahead, “the advent of 5G for remote field production will be a significant development,” said Kelly. “This will potentially allow high quality content to be delivered via a single SIM connection in the field. With the advent of HEVC/H265 encoders – available for JVC’s Connected Cam now – highly cost-efficient field content production will be possible.” As cloud-based editing and content delivery systems develop, he believes “it will be possible to create live production content with much lower initial investment, further broadening the range of content available.”

“We see cloud-connected camera technology evolving in future to include more metadata, and as more bandwidth becomes available, higher resolutions and bit rates being delivered (contribution and distribution), which will enable UHD at higher resolutions and bit rates while also accelerating emerging workflows such as VR,” said Thompson.

Ray thompson Avid

Making the connection: Avid’s Ray Thompson

As companies accelerate the move to the cloud post Covid-19 it won’t only be with connected cameras but connected encoders, mobile devices, apps and more (an internet of things for media). Avid hopes to accept any incoming streams regardless of protocol and also enable distribution whether from the cloud or hybrid deployment. Thompson believes this will reduce the barrier of entry, especially for digital-first media companies. “This will also lower overall costs (vs. satellite and fibre), lessen the amount of people required to be on site, also lowering costs, while driving much higher efficiency throughout the production value chain accelerating delivery and taking advantage of automation and orchestration.”

The pandemic has “seen a forced acceleration of the adoption of cloud-connected cameras, due to their ability to allow more remote operation with fewer crew,” said Hedley. “Although broadcast infrastructure is still catching up to the consumer world in terms of connectivity, we envisage that many of the changes that have been implemented during recent months will remain in place. The only area where advances are a real necessity is the availability of 5G networks as this will allow greater flexibility and result in cloud-connected technologies becoming more readily accessible to all.”

The move to IP and cloud production “will be huge,” added van Geel. “If you don’t make use of it, you will look ancient real soon, most likely, in the broadcast industry.” With the need to produce more content with fewer resources, and more working from home, the cloud allows users to “make sure that multiple programmes have the same look, because you can really oversee more.” Indeed, he believes that the quality doesn’t need to be lower because more tools will get automated. “If you invest in the wrong things now, you might have a hard time three years from now because you didn’t pay off for the equipment completely yet,” which means equipment has to be more flexible, and “virtualisation is the key to flexibility,” he said.