Contextual AR in the wild: enhancing live cycling broadcasts

Abstract

Advanced computer vision and machine learning are transforming sports broadcasting. While augmented reality has successfully enhanced fan engagement in controlled environments like soccer and basketball stadiums, its application in challenging outdoor sports like cycling is virtually non-existent. Over the past two years, we focused on bridging this gap, developing a fully automated workflow for virtually enriching cyclists and other key objects in live race broadcasts.

The first part of our work automates rider and team detection captured from 
diverse viewpoints. Combining modern object detection, Siamese networks for similarity learning, and advanced tracking offers a robust solution for cyclist identification and tracking. The second part of our work explores the use of 3D pose estimation of cyclists to create augmented reality visualisations. Finally, we also focus on the development of an automatic pipeline for POI detection, recognition, and visualization, which facilitates the 
graphics generation, a process that was previously done manually.