Presentations and papers: All of this year’s Technical Papers

5G - Delivering on the promise

Scratch below “5G is faster” and you’ll uncover a technically sophisticated framework with detailed architectural solutions designed to support many new use cases and business models. In this session we look at two extremes - edge computing, enabled by 5G, which promises highly customised and computationally demanding media experiences to individual users – but what do you actually implement and how do you deploy and manage this extraordinary degree of customisation – our author has been doing just this.

At the other extreme, with the standardisation of 5G Terrestrial concluded, the convergence of mobile and broadcast is finally done! Here, our author argues further changes including a new radio design may still be needed for viable commercial operations. Finally, our supporting paper provides an update on the status and on-going work in 3GPP’s Media Streaming, Broadcast and recent release 17 work on new codecs and edge computing.

• Read the papers and watch the sessions here

Facial recognition - Various facets of a powerful media tool

Facial recognition is one of today’s most controversial media technologies. Recent claims are that it can reliably recognise subjects wearing sunglasses or medical masks, and that it can even differentiate between identical twins.

Not all of its many possibilities are sinister, however. In this session we shall see how facial processing is already employed by a global broadcaster and that it is demonstrating its value in several areas of live broadcast production, such as assisting a commentator to recognise each of 500 runners in a road race.

In a second presentation we shall discover how a convolutional neural network has been trained to classify facial expressions, allowing it to recognise and tag emotion in video material. This provides an entirely new way of categorising and searching drama and movie content.

• Read the papers and watch the sessions here
  

More formats, more conversions

Whilst the enhanced video formats of Ultra High Definition (UHD), Wide Colour Gamut (WCG) and High Dynamic Range (HDR) present in spectacular quality – they also bring a myriad of format conversion challenges, whether that is up-converting legacy content for use in new productions or down-converting HDR/WCG to suit traditional devices.

In this session, we address both of these challenges – one demonstrating an effective colour mapping model that takes into account the behaviour of the human visual system and the other using machine-learning for super-resolution in a production environment. Our supporting paper continues the enhancement theme assessing the effectiveness of machine learning to reduce coding artefacts.

• Read the paper and watch the sessions here

Advances in Video Coding

In this session, IBC showcases significant coding gains arising from both traditional and artificial intelligence based techniques. 

Quantization is at the heart of compression and in this masterclass paper, you will learn how practical advances in rate-distortion optimisation continue to drive encoder gains, across codecs. Then there is the rise of artificial intelligence which is playing an increasingly important role in video compression. Here you will hear how machine learning has been successfully used to identify “salient” areas of a picture for concentrated bit allocation, but is it robust and predictable?

Our supporting papers continue the artificial intelligence theme addressing, where in video compression it is (and is not) best applied and how it might be standardised.

• Read the paper and watch the sessions here

The Cloud - For Live and Production Workflows

Cloud-based production is revolutionising the working practice of journalists and entertainment media producers, allowing increased flexibility of location and opportunities for innovation and speed of creation. In this session we explore two advances which contribute to the efficiency of the workflow.

First we look at the development of an app-based news service where journalists not only create the stories but drive the whole content creation and delivery processes. The design must therefore focus on ease of use with intuitive interfaces. 

Our second cloud presentation delves deeper into the technology involved in developing a hybrid premises or cloud-based system for live and production workflows. 

This session also recognises an additional paper on cloud-based multi-station architecture for networked radio stations, which forms part of the conference proceedings but could not be presented personally for scheduling reasons. This paper is highly regarded and is recommended reading.

• Read the paper and watch the sesion here

Advances in Audio - Using some remarkable signal processing

The subject of advances in audio is explored by Rob Oldfield in his paper ’Cloud-based AI for automatic audio production’ and by Matteo Torcoli in the paper ’Dialog+ in Broadcasting: First Field Tests using Deep-Learning-based Dialogue Enhancement.’

Every broadcaster knows that the most common complaint from viewers is that programme dialog is hard to discern against a background of atmospheric sounds, mood music and competing voices. It is especially a problem of age, where 90% of people over 60 years old, report problems.

• Read the paper and watch the session here

Cutting Edge Technologies - A preview of some experimental concepts

In this session we shall examine three new ideas which are thought -provoking and potentially media game-changers.

First, we introduce the robot companion who watches TV with your family and friends, and shares the fun and the emotional engagement. 

Our next on-going experimental development concerns a live performance where musicians in a concert hall will play together with itinerant musicians who are walking through the streets of the city. 

The third initiative is a novel means by which an individual’s entire media streaming history, from all their service providers, can be used to provide personal recommendations without any contributing service needing to hold or process their data.

• Read the paper and watch the session here

Orchestrated Devices - A vision of networked home entertainment

The number of devices in the modern home which are capable of reproducing media content is already considerable. However, a user will typically employ only one when enjoying a particular form of entertainment.

In this session, we shall hear how a prototype audio orchestration tool has been designed and trialled on several productions to evaluate the principle of creative orchestration; with extremely positive results. In a second presentation, intelligence within the home automatically orchestrates the incoming media across the available devices taking account of the content, the environment and the wishes of the user. Trials of this architecture will also be seen to exceeded the expectations of users.

Our supporting paper investigates the related challenge of orchestrating the desires of multiple users sharing devices.

• Read the paper and watch the session here

AI in Media Production - Creating new markets for linear content

For decades, broadcasters have been producing linear programmes, such as news, magazines or documentaries, which contain valuable audio-visual information about a vast variety of individual topics. The problem is that these individual topics are often neither addressable nor findable.

Could AI and machine learning, segment or chapterise this archived material so it would be re-usable in the interactive digital world? Might AI even be able to re-edit it into personalised media? We look at a fascinating project which is doing all this and more. Improvement is still required, especially in the editorial challenges for AI, of creating re-compiled media, but public-facing trials are underway and generating much interest.

• Read the paper and watch the session here

Optimising Streaming – Savings at Scale

Streaming is ubiquitous and both bandwidth and storage hungry. In this session we focus on improvements to both of these challenges, a must for cost saving at scale.

Our first paper investigates, tests and provides an open-source solution that optimises viewer experience when adaptively streaming context aware encoded video, a particular challenge due to its bursty bit profile.

While our second paper addresses the storage challenge of supporting both HLS and DASH low-latency streaming, with clearly illustrated examples and experimental results. In our supporting papers we continue the optimisation theme as well as a seeking to address the “watch-together” synchronisation challenge.

• Read the paper and watch the session here

ST2110 on Modern IT Infrastructure – How Difficult is it?

The goal of leveraging modern high-performance IT fabric for professional media handling is laudable. But just how difficult it is? And how close to commercially off the shelf (CoTS) IT equipment are we?

In this session, you will learn from an expert who lays bare their practical experience of the complexities and challenges of implementing ST2110 and asks whether this is the right solution to achieve the goal. In our second paper the author demonstrates a state-of-the-art GPU/DPU in a Microsoft Windows device outputting ST2110 to a networked attached display.

• Read the paper and watch the session here

XR Producing Immersive Experiences

For XR to come of age, we need practical capture systems, familiar production processes and an effective blending of game and multimedia technologies. 

In this session, we address all three of these challenges: showcasing a production environment with 6 degrees of freedom (6 DoF) video and spatial audio capture composited using a game engine, a practical real-time (30fps) point cloud capture system – as well as discussing the current state of convergence between game and traditional multimedia in XR systems.

• Read the paper and watch the session here