Abstract

This paper presents the world’s first implementation and release of the upcoming MPEG standard for video-based point cloud compression (VPCC) on today’s mobile hardware. As V-PCC offloads most of the computational burden on existing video coding solutions, real-time decoding can be achieved on essentially every single media device on the market. Furthermore, as the infrastructure for 2D video distribution is well established, distribution and storage of V-PCC content can already be achieved on today’s networks.

Introduction

Due to the increased popularity of augmented and virtual reality experiences, the interest in capturing the real world in multiple dimensions and in presenting it to users in an immersive fashion has never been higher.

Volumetric visual data represents dynamic 3D scenes and allows a user to freely navigate within. Unfortunately, such representations require a large amount of data and uncompressed transmission is not feasible over today’s networks. Therefore, the Moving Picture Expert Group – MPEG (ISO/IEC JTC1/SC29/WG11), as one of the main standardization groups dealing with multimedia, recently started an ambitious roadmap towards immersive media compression. One of the elements of this roadmap is a standard for compressing dynamic 3D point clouds. This emerging standard, ISO/IEC 23090-5, will rely heavily on the use of already available 2D video coding technology. Thus, claiming superior compression efficiency and accelerated time-to-market.

Following this claim, this paper presents the world’s first implementation and source code release of the upcoming MPEG standard for video-based point cloud compression (V-PCC) on current mobile hardware, as shown in Figure 1. Because V-PCC offloads a lot of the computational burden on existing video coding solutions, hardware video decoders, available in essentially every single media device on the market, can be utilised for real-time decoding. Furthermore, as infrastructure for 2D video distribution is well established, such as ISOBMFF and DASH, these existing solutions can be easily used to support distribution and storage of V-PCC content. In this paper, we will address topics such as V-PCC compression efficiency compared to state-of-the-art, decoding and rendering capabilities on current mobile clients, as well as approaches for V-PCC content distribution over existing 2D video infrastructure.

The remainder of the paper is structured as follows: First, we provide a background overview on volumetric video compression. Then, we will present the upcoming MPEG standard for video-based point cloud compression (V-PCC) in its latest form, before going into details on the decoder implementation for real-time augmented reality (AR) playback, including performance evaluation on current mobile phone technology. The paper will conclude with an outlook on ongoing standardisation activities, such as V-PCC file encapsulation and streaming for distribution and storage.

Download the full paper below