Crafting and delivering coverage of the Fifa Russia World Cup required robust workflows, and provided Fox Sports with some valuable lessons for the upcoming Fifa Women’s World Cup.
When comparing the upcoming FIFA Women’s World Cup to the men’s tournament which took place in Russia last year, Brandon Potter, director of post-production at Fox Sports says this year’s tournament will be different.
“In Russia, our entire playout facility was in the IBC. This time that facility is back at home in LA. We are able to do more locally and have to send less people to France.”
Last time, 99% of editorial was done remotely, and Levels Beyond with its Reach Engine management platform and Aspera were crucial to what Potter calls “a pretty innovative workflow” for getting content back to LA for fast cutting.
“In a way, the Russia World Cup was a proof of concept and dress rehearsal for France. We used many new technologies that had never been implemented before, but by proving all those systems in successful workflows we are now using that for the women’s tournament,” he says.
“One of the challenges is 1080 50p, because our infrastructure in the US is 720, and we broadcast at 720 59.94. The FIFA in-house HBS Production Department is giving us all our content and feeds in that format, and then we will have a couple of roving venue kits and a couple of ENG crews. There will be remote ingest with the roaming venue kits plus one at the set location, so anyone can drop by and pop in a media card.”
In Moscow, Fox Sports generated two petabytes of content and even fed in 100TB of content in a single day. It also took a total of six feeds at once but in France it will probably be writing 12.
“Maybe there is a lack of cohesive monitoring generally speaking in our industry” Brandon Potter, Fox Sports
“Technically we could write as many files as we wanted as long as we have the bandwidth,” says Potter.
“We are going to implement the same technologies that we used for Moscow – Aspera, Telestream and Reach Engine to move the content around.
“All the file-based content, camera card material and ENG source content will be generated mostly by us and not by HBS, so we bag and tag it and present it and use Aspera to get it up to the cloud quickly via accelerated file transfer. We are then able to present proxies of that content to end users,” he added. “They can poll and pick what they need so we don’t have to move everything down from the cloud.”
Writing priority feeds
The HBS feeds were and will be presented at 1080 50p for editorial. In Moscow Fox Sports handled around 60 channels.
“We presented all of that into Reach Engine with an HLS (HTTP live streaming) proxy that users can access for a clip or a file and get content where they need it to go quickly. We also had the Aspera FASP Stream that was writing what we deemed as priority feeds,” says Potter.
“As verification of concept we have been able to stream three feeds at once without any sort of delay.“
Potter sees one issue that is close to resolution. He says: “If I want to monitor my file, or it I want to take action on a file whatever the system, and I want to send it to somewhere, I probably have to go to four or five status monitoring tools to actually see where my file is.
“This is a challenge, and I don’t know that anyone can solve it at the moment given all the different pieces, because to a certain extent one system stops looking at it unless they all talk to each other and say my process is done and I have thrown it over the fence,” he adds.
Potter could check Reach Engine first and see he instituted a file transfer, and can track that it passed via Aspera to EVS.
“Reach Engine is getting better with its API and we are working better with our API; integrations will show status monitoring in one tool,” he added. “But maybe there is a lack of cohesive monitoring generally speaking in our industry.”
Rinse and repeat
Fox Sports works in a hybrid environment, meaning it is not solely working in the cloud.
“I don’t think we are in a place to really work using that model to properly edit videos, so we are still living in that world where we get sent stuff up and pull it back down. We are not airing content out from the cloud, but I think the industry is getting there,” says Potter.
One day editing and broadcast will be done in the cloud, but the workflows implemented for Moscow then Paris are about editing on premise and accessing content in the cloud, which is then moved to where it needs to go.
- Read more Interview: Reed Kaufman, Fox Sports
Potter was at the Fox broadcast facility in Charlotte testing all the kit, workflows and connections.
“This year is a little bit easier because it is somewhat rinse and repeat. We have already proven the workflows, so it is just tweaks here and there,” he said. “We are trying to establish an out of the box solution for all our dual events. As we start to Frankenstein these peaces together, and as we grow and learn and have more time to do things like archive on the fly, we will adopt and implement new technologies. This is a thing we can do from event to event.”
“The lines become a bit blurred when you have the attitude that no one wins if any critical part of a given workflow breaks down. It really doesn’t matter who is at fault” Michael Flathers, Aspera
Aspera chief technology officer Michael Flathers prefers the phrase ‘pushed the envelope’ to ‘proof of concept’ as he looks back at Moscow and ahead to France.
- Webinar Join IBC365 and Aspera to explore Blockchain: A framework for the secure exchange of digital assets
He says: “We pushed development during the event in ways we hadn’t previously imagined. I give Fox Sports a lot of credit for wanting to move beyond status quo and being willing to set aside time to experiment along the way.
“We had goals that we had set with regard to things like the number of seconds behind live (camera to being editable within Adobe Premiere), and in the end we were more than three times faster than targeted. For these workflows we were dealing with fairly high bit rate streams of approximately 250Mbps for video and audio combined,” he adds.
“There’s nothing like the live event itself. You simply cannot anticipate everything that will happen during a multi-week tournament. We were very fortunate in that Fox Sports and all the vendors involved worked together to ensure success rather than pointing fingers.”
There were instances where Aspera realised that it could reduce the risk of human error by writing code to more tightly integrate all the vendor technology.
“The lines become a bit blurred when you have the attitude that no one wins if any critical part of a given workflow breaks down. It really doesn’t matter who is at fault. This attitude allowed us to do much more that we otherwise would have been able to do,” says Flathers.
Leaving men’s football behind elicited the recognition that some ad hoc things had to be dealt with, and fast-forwarding to France, Flathers knew Aspera could make the same streaming-based contribution.
“The same rigour with regard to the staging environment is being carried out but it doesn’t feel as experimental,” he says. “The success of Moscow gave us a new baseline and allows us to push things even further including such elements as increasing the number of high bit rate edit at home feeds as well as introducing additional workflows such as the use of Aspera streaming technology to essentially ‘auto-archive’ content to the cloud at the same time as those feeds are being sent to Los Angeles for live editing.”
This will save a good deal of time overall, reducing or eliminating the need to archive that content post-match, and it also includes the ability to do ‘fan-out’ delivery in real-time. Will Aspera’s new Connection Manager play a role in France? This technology will automatically tie streams together, providing acceleration where needed while allowing users to see the status of those connections.
“It is certainly something we are excited about but it is not actually going to be used in production workflows for the FIFA Women’s World Cup,” says Flathers.
“As the technology responsible for transporting data, in many ways we essentially tie the other technologies together. Fortunately, we have a rich set of API’s that have allowed very tight integration between our technology and that of both Telestream and Levels Beyond. They leverage the IBM Aspera API’s heavily to make the handoff seamless.”
- Read more The TV set triumphs at the World Cup