The industry debate between storing data in the cloud versus on-prem is a long running one, and as an expert panel in a recent IBC webinar discussed, one size does not fit all, writes Mark Mayne.
From the intricacies of strategic cost management to the demands of secure data storage, scalability and accessibility, there are plenty of variables to consider in refining a storage strategy that works for any individual business, whether national broadcaster, indy producer or rights owner. The participants in the recent webinar, Storage: On-prem vs. Cloud covered off many of the key variables, as well as highlighted some overall industry trends.
Interestingly, there was a crucial tipping point exposed by the accompanying webinar polls, with a substantial swing between voters’ current storage profiles (mostly on-prem at 42%), to the future, where a move to hybrid (53%) seems the order of the day.
The first question was: ‘What approach are you currently using for storage?’ Responses broke down into Mostly cloud 11%, Mostly on-prem 42%, Mostly hybrid 23% and finally A mixture 23%. The followup question, ‘What approach to storage would you like to use in future?’ exposed a substantial shift, with the same categories seeing significant changes - Mostly cloud 25%, Mostly on-prem 8%, Mostly hybrid 53%, and A mixture at 12%. Overall, hybrid winning out with a substantial 30% increase in the future over the current architecture allocation.
Hybrid storage defined
Conrad Gouws, Technical Architect, RTE stepped up first to define hybrid storage in a broadcast context: “For me when it becomes hybrid is when you start using that ‘valet parking’, where[data] is in many places or in all places, but it’s still it’s all managed by one layer. So we can - without the user knowing - make decisions like not to restore from one frame when someone asks for their file. We’ll take the hit and you can restore it from AWS because we’re moving the servers or upgrading, or the system is down. For me, that’s when it becomes hybrid.”
Russell Grute, Managing Partner, Broadcast Innovation agreed: “Designed around the enterprise, the cloud is not a location, not a place. It’s a technique. It’s a set of processes which you want to use commonly, whether you’re operating on prem or accessing and activating media and data away from your enterprise.”
Augmented Disaster Recovery
A key aspect of cloud storage is the inherent ability to offset localised data loss, specifically in the case of disaster, although there are many other areas where local data can be seen as a potential risk. However, the panel began by focussing on the wider question of Disaster Recovery, with Gouws picking up the baton: “I mean, we look at business continuity. Learning from decades of heavy engineering in television paths and disaster recovery on this one specifically, that is the reason we built it this way. So anything that is critical, or of extreme importance, there are multiple copies not just stored on our system, [but also] stored in the cloud vendors.
“That is how we manage the ‘three-two-one’ golden rule for archive storage. The system writes it in the native format of the provider of the public cloud providers, so if that system went away, you could still go to [the cloud provider] and just retrieve the files. I hesitate to say it’s bulletproof because nothing’s bulletproof, but…”
Grute followed on: “Our industry was raised on the fire drill, where you only tested when something goes wrong. These days - thank you to the IT industry - everything should be tested in actuality, and then you use half of it all the time. So whether that’s the reserve circuits or back end of the network or whatever, but also with staff on the move not at their regular desks, connecting to different parts of the network with different tools, permissions - these test things in a great way.”
Data security an essential
Grute highlighted that security is a key concern - inevitably - but something that needs to be managed with a certain level of risk acknowledgement.
“To some extent it’s an arms race. On Prem with your own internal team, how hard do they want to work and keep up, vs how much can you trust a third party, best of breed cloud vendor of any service and their particular policies? But you can’t be blind. You need to understand security at a user level and at an enterprise level. Whether you [then] contract it out to a subcontractor or whether you deal with it internally you must understand it.”
Gouws: “They’re only as secure as they’re configured. So you either outsource your configuration to somebody else or… But if you have a key leak somewhere in some code on GitHub, someone’s going to get access to it on the public cloud vendor will just say sorry, it’s just not our problem. There’s always the mix between what we can afford and what we would like…”
He continued to raise another particularly crucial point around outsourcing any function, but particularly cloud: “The other question we always ask ourselves, what are the priorities for this public cloud vendor? We are not a BBC. We are a small public service broadcaster. If the entire AWS stack is in some way compromised, where will we be [priority-wise] versus any of the tech giants that are on that stack?”
Economics and scalability
Gouws was keen to emphasise - especially from the perspective of a publicly-funded broadcaster - that the cost of cloud storage is entirely predictable. “In the parts of our solutions that go into the public cloud, costs are very predictable. We’re a publicly-funded organisation, so we have tendered these things. We know what the cost will be for anything that we store, and we can predict that accurately. It’s the same team that manages on premise storage, that also manages the relationship with the various cloud vendors, and they’re very good at it. So - no unpredictable bills, but it does take a lot of effort and planning.”
Grute responded in kind: “Engineers have to deal with more spreadsheets than ever. They have to manage this cost modelling. Also the vendors trade on low latency availability, that’s their premium. But systems can be designed to wait, and proxies are good enough for many applications which you could hold much closer to your CDN or to to the edge of the network. But overall, the unpredictable egress is a [potential] biter, particularly for the first few cycles if you haven’t negotiated well…”
Skills a crucial element of the hybrid mix
Gouws also highlighted the importance of people and skills - as is the case in any engineering situation: “Our headcount has stayed largely the same since 2013, but [cloud] does change the skill set. I can tell you the first time we introduced people to the virtual servers, the first question was, where’s the power button to turn it off? So it’s been a journey!”
Catch the full webinar, Storage: On-prem vs. Cloud on demand now, and get the full range of insights from the expert panel. Alternatively, check out the full on-demand IBC webinar catalogue and/or register for any upcoming IBC 2023 webinar topics that appeal.
Read more IBC365’s 2023 webinar programme
No comments yet