In common with many industries, TV and video production is likely to be transformed by Artificial Intelligence (AI) and Machine Learning (ML), with software and algorithms assisting production tasks that, conventionally, could only be carried out by people.
Expanded coverage of a diverse range of live events is particularly constrained by the relative scarcity of skilled people, and is a strong use case for AI-based automation.
This paper describes recent BBC research into potential production benefits of AI algorithms, using visual analysis and other techniques. Rigging small, static UHD cameras, we have enabled a one-person crew to crop UHD footage in multiple ways and cut between the resulting shots, effectively creating multi-camera HD coverage of events that cannot accommodate a camera crew.
By working with programme makers to develop simple deterministic rules and, increasingly, training systems using advanced video analysis, we are developing a system of algorithms to automatically frame, sequence and select shots, and construct acceptable multicamera coverage of previously untelevised types of event.
Artificial Intelligence (AI) and Machine Learning (ML) have the potential to increase substantially the range and scale of events that broadcasters and other content producers can cover. It is not clear what the timescale and impact of these technologies will be, or the extent to which they will assist existing human craft roles rather than automate parts of them.
In this paper, we present our first efforts to investigate these opportunities. Our recent work to simplify the process of covering staged events such as stand-up comedy or panel shows using new software tools and novel craft workflow is described: the BBC prototypes Primer and SOMA use web technologies and our IP Studio implementation of the AMWA NMOS standards to allow a single operator to produce “nearly live” coverage of such performances.
We then describe our experiences in developing Ed, a system that attempts to automate the work of this craftsperson using a rules-based AI approach. The challenges associated with evaluating the performance of such a system are discussed, as well as the prospects for improving it using ML.
Our objective in developing automation for a specific production workflow is to learn where the limitations of AI lie, in the expectation that our industry will benefit most from AI and ML in the short term by using these technologies to make people more effective—automating their most time-consuming or repetitive tasks—rather than by supplanting them.
by Craig Wright, Jack Alnutt, Rosie Campbell, Michael Evans, Ronan Forman, James Gibson, Stephen Jolly, Lianne Kerlin, Zuzanna Lechelt, Graeme Phillipson & Matthew Shotton