HDR for the second screen

No comments

ABSTRACT

High Dynamic Range (HDR) will soon be just as much about delivering great video experiences to tablets and smartphones as it is about delivering to the big UHD TV in the living room. For video service providers, however, the challenges are different.

UHD HDR content is crafted with the big screen in mind. As a result, the small localised highlights and spatial details of HDR content can be obliterated when it is scaled, filtered, and encoded using adaptive streaming protocols to fit handheld screens.

This paper provides quantitative methods to optimise HDR for the second screen and adaptive bitrate services. We provide methods to measure and mitigate distortions of overall HDR luminance variations as well and the localised highlights and spatial details that are unique and significant in each video frame. A key take-away is that we provide methods to select the best combinations of bitrate and encoded resolution to use in multiresolution adaptive streaming of HDR content.

INTRODUCTION

Adaptive streaming is rapidly becoming a dominant method for distribution of television to all screens from the big screens in living rooms to smaller-screen smartphones and tablets. At the same time, displays big and small are becoming much more capable of rendering the deep darks and bright highlights that make HDR special.

Together, adaptive streaming combined with advanced display technologies can enable HDR to become a high-value second screen experience. Ideally, we would like to create a consistent HDR experience across all screens. A challenge is that small screens tend to have lower resolution than big screens.

They also tend to be used more often at the edge of lower-bandwidth wireless networks. Adaptive streaming protocols compensate for these issues by making several variants of media content available to adaptive streaming clients. The set of variants, often called an adaptation set, is made up of several representations of the original content at lower resolutions and at different bitrates.

Adaptive streaming clients select a variant from those available in the adaptation set that makes best use of the client’s bandwidth availability and rendering capabilities. As bandwidth and other conditions fluctuate, the client can adapt by selecting different variants.

The main advantage of adaptive streaming is smooth uninterrupted playback. For HDR, however, there are potential negative side effects related to the loss of spatial detail and localised highlights when clients select lower-resolution lower-bitrate variants.

The key question for service providers is this: Can we design and construct adaptation sets in such a way as to minimise HDR distortions and promote consistency across screens? This paper provides methods to help achieve that goal.

This paper provides methods to quantify HDR distortions, take steps to mitigate those distortions, and select the best combinations of bitrates and resolutions to include in HDR adaptation sets.