The infinite capacity media machine

No comments

ABSTRACT

The fast-paced developments of cloud computing are evolving new kinds of machine that will be applied to professional media production.

Serverless, distributed, clustered and GPU accelerated, the nature of cloud-fit architectures is changing the patterns and design of software and services. Is it even possible to shift the highly sequential, line-timed infrastructure of existing media facilities into the stochastic, message-driven and asynchronous cloud? What new creative possibilities emerge from a platform that scales to enable personalised production?

The paper imagines the kind of machine you would build if you had an infinite amount of capacity in terms of networks, storage and compute.The design for such a machine and a plan for how to build it are introduced in the form of the Agile Media Blueprint.

In the context of video data, measurements and analysis are provided for and of the capability of current systems in relation to the idealistic notion of infinite capacity.

INTRODUCTION

Driven by trends such as big data and social media, aggregate performance of information technology is advancing rapidly. Depending on your preferred model (e.g. Moore’s Law, Gilder’s Law), networking, compute performance and storage capability (bandwidth and capacity) double every six months to three years.

In tandem, trends in software development and cloud are reorganising the way that compute resource is configured and accessed, going beyond the boundaries of single computers to exploit massively parallel Graphics Processing Units (GPUs) and compute clusters.

A media company that is planning its future technology platform based on IT trends needs to consider the potential capacity of the machine available at deployment time. Is the rate of performance increase of IT technology outstripping the rate of increase of data rates for media quality, even trends from SD to HD to 4K to 8K with higher frame rates and increased colour depth?

If this is the case, consideration should be given to an idealistic machine with the capacity to process as much media data as required, for any given workflow and in a time tending to zero, termed here the infinite capacity media machine. This paper starts by characterising an infinite capacity media machine in terms of media formats, network, compute and storage.

Using technologies available today, the Agile Media Blueprint [1] - a technical plan for how such a machine could be built today – is introduced. To test how far along the road generic Information Technology (IT) is in delivering the idealist machine, some measurements of media data transported with commodity networking, processed with GPUs and managed by clustered in-memory caches are presented.

The gap between idealistic work and infinite capacity is then analysed, with a description of future work to build the machine and bridge the gap.