This Accelerator project, led by the Associated Press, addressed a challenge that is a largely manual and very costly process for media organisations today; producing shot lists of raw and edited video content, with written depictions stored alongside the video in the archive.

Champions: Associated Press (Project Lead), Al Jazeera Media Networks, ETC.

Participants: Vidrovr, Metaliquid, Limecraft

By exploiting some key capabilities of AI, using face, object and voice recognition plus sentiment analysis and machine learning, the team developed a solution and demonstrated an impressive proof of concept showing how that process can be fully automated. Marshalling the full suite of AI layers from speech-to-text transcription to facial, object and voice recognition, alongside semantic and sentiment analysis; and applying those to specific training sets of video assets, allowed the team to build a model to address the challenge.  

While much work is still to be done in this area, the project has made great strides in solving one of the biggest headaches for news agencies and broadcasters; replacing the labour-intensive process of producing shot lists and emancipating video producers from tens of thousands of hours spent manually creating shot lists in written narrative form.  

Why AI for automated shot list creation?
AI is already applied to video - tags assist search and discovery, semantic and sentiment analysis enrichment is widespread - but no single solution exists to replace the human effort of creating a shot by shot, narrative depiction of the asset, thus freeing producer time for more creative work. Vital creative resources are diverted to this semi-skilled, though nonetheless essential task.

Without a proper shot list there can be no understanding of what is contained in raw and edited video assets and therefore no ability downstream to retrieve or commercially realise the value of footage stored in archive.  

For further information: accelerators@ibc.org