How do audio engineers develop an expertise in surround mixing and what are the elements that characterise a great immersive mix? David Davies speaks with the best ears in the business to discover more.
Along with 4K and HDR, the rise of immersive audio – led by Dolby Atmos and MPEG-H – has been one of the defining production technology stories of the past decade. From cinema releases and streaming series mixed in Dolby Atmos, to installations and live events being enhanced with different configurations delivered in multiple formats, audio is an increasingly immersive endeavour across application types.
Identifying a tipping point for a technology is not always easy, but in this case, one can pinpoint the mass availability of immersive audio music, via sites such as Apple Music and Tidal, as an important moment. But despite the increasing footprint of surround music in both professional and consumer domains, it’s arguable that – in relative terms – immersive audio mixing is still in its infancy as regards ‘standard practices’ and a commonly-shared notion of what constitutes a ‘great immersive mix’.
An immersive odyssey
Robert Edwards has been a leading television sound supervisor and audio mixer for five decades. Sound Director of Video Sound Services Ltd (VSSL) since 1984, Edwards’ numerous mixing credits include the World Television sound feed for the opening and closing ceremonies at the Olympics in Brazil and Tokyo, the Eurovision Song Contest, and British entertainment staples such as Harry Hill’s TV Burp, The Masked Singer and Michael McIntyre’s Big Show.
Read more Pitches unveiled for Kickstart Day 2024: projects to address biggest challenges in M&E
Edwards traces his engagement with surround audio production back to the turn of the millennium when he was involved in developing “some very basic Pro Logic-based sound coverage” for Sky football coverage: “Essentially, it was stereo but with the addition of rear microphones put in out of phase.” After 2006, VSSL was more regularly producing 5.1 mixes for clients including Sky, HBS and ITV1, where the experience of mixing The X-Factor in this format – for internal reference as opposed to domestic consumption – was particularly invaluable.
Having had the opportunity to experiment with different approaches during this “long gestation period” ensured that Edwards was well-placed to hit the ground running when Nuno Duarte – Sound Design and Audio Manager at Olympic Broadcast Services (OBS) – announced that a 5.1.4 immersive audio configuration would be implemented for the Summer Games in 2020. “There was no specified format; instead, they made the decision that the raw streams would be sent to [the rights-holding broadcasters] and they could then encode the audio in whichever way they wanted,” says Edwards.
For OBS, Duarte – who praises Edwards as one of the “pioneering” mixers in the field – says that the core approach revolved around an expansion of some techniques it was already employing to “capture audio in a three-dimensional way, then working in baseband to create the 5.1.4 immersive that gives us the experience and [enables] it to be delivered to the rights holders, who can then select which platform they want to use.” It’s an “agnostic approach” that continues to define OBS’ vision for immersive audio as another Summer Games looms on the horizon.
Having witnessed a sequence of significant transitions in broadcast audio mixing – beginning with the shift from mono to stereo – during his long career, Edwards is understandably pragmatic about the advent of immersive. “I’ve been doing this for 50 years and come with a background of doing all sorts of mixing for all sorts of events, so I have a very open mind about all the technologies involved,” he says.
Nonetheless, Edwards indicates that “breaking [the mix] down into chunks” – always an important consideration with challenging live productions – is more important than ever. “With an event like Eurovision, you are certainly juggling more than 200 input sources at once, which means you have to plan how you are going to deal with them in a live environment. That means creating smaller groups and then sub-mixing stuff so it becomes [more manageable].”
Whilst he is effusive about the creative opportunities already afforded by immersive audio, Edwards implies that it is likely to be with the advent of increased personalisation – i.e. enabling audience members to customise the mix in line with their own preferences – that the technology realises its full potential. Ongoing advancements in the utilisation of metadata will be critical here: “It could be something pretty basic [to begin with], such as giving control of the metadata that allows you to decide where you want to be situated within the stadium and, for instance, hear the sound as if you were an away fan sitting in the crowd. Or perhaps you are hard of hearing and would like to change the location or increase the volume of the commentator… [with personalisation] you have these sorts of options.
Edwards also expects that the underlying technologies – and the way they are deployed by broadcasters – will continue to develop. There are already some appealing motifs in film and TV production “such as when you hear rain falling above your head; that’s a really nice immersive effect. So whilst I don’t think we’ll ever reach a ceiling for what is possible, I do expect there to be a maturity about what immersive is really good – and not so good – for.”
Focusing on the details
As both a working freelance audio mixer and an academic at the Darmstadt University of Applied Sciences in Dieburg, Germany, where he tutors the next generation of TV sound engineers and supervisors, Felix Krückels is necessarily engaged with new technologies in every conceivable sense. And like Edwards, his first experiences with immersive audio were for high-profile sports tournaments – in this case, the 5.1 mix of the FIFA World Cup held in Germany in 2006.
Krückels believes his extensive experience as a classical music Tonmeister served him in good stead as he began to navigate the immersive era. “If you do recording for classical music, you are always looking for a lot of [immersive qualities] in stereo already, including obtaining a lot of details from spot microphones,” he says. “I actually took the same kind of approach [into sports] when I started mixing in 5.1, so the result is that I felt very comfortable with immersive from the beginning.”
Read more Spatial computing and immersion - a new era for volumetric production
Sports have remained a core element of Krückels’ immersive mixing, and he has found boxing events – where there is an imperative to achieve a compelling audio experience that balances background music, crowd noise, announcers, commentators and the often highly-dynamic action inside the ring – to be an especially interesting challenge. But ultimately, it all comes back to the rigorous capturing of sonic detail, especially in support of the moments that define a match.
“You can create a really huge immersive production, but if you can’t hear the ball being kicked or the boxer discussing his tactics with the coach during the break, then it’s not going to be a good broadcast show,” he says. “So all of these ingredients have to come together effectively, and in a way that means [the audio] is as ‘big’ as the visuals.”
Krückels does not understate the impact that the addition of immersive has had on the amount of deliverables required for tournaments and events, although advances in mixing technology have made it much easier to “hear everything in parallel”, such as for national and international feeds. Looking forward, he implies that more powerful automation will also help to lighten the load: “It may be that in the future, clever software provides the ‘second pair of ears’.”
Analogue ethos in a digital age
Music producer, engineer, mixer and artist Stephen W. Tayler is currently at an intriguing juncture. Having been involved in 5.1 mixing for nearly two decades, Tayler is “just beginning to get involved in Dolby Atmos”, using a commercial Atmos space at Real World – the recording complex near Bath owned by Peter Gabriel where he has long had his own personal studio – known as the Red Room.
Having mixed a live concert release in 5.1 for Howard Jones in the early 2000s, Mark Powell of Esoteric Recordings and Cherry Red asked Tayler if he would be interested in remixing some classic albums for the (then still relatively) new format. An initial project for British rock band Be Bop Deluxe ultimately led to remixes of five of their studio albums and a live set – the positive reaction to which resulted in similarly long-running associations with Camel, Van Der Graaf Generator and many others: “People started to be interested in what I was doing, and since then I’ve been having the time of my life!” he says.
Today, work takes place entirely in the box using Pro Tools and a speaker configuration based around Genelec 1073s. “I’ve essentially developed a workflow that is based on my analogue days, and as software has developed I’ve been using more and more emulations of vintage gear from the period when I started,” says Tayler, whose career began at London’s celebrated Trident Studios in the 1970s.
Maintaining a structured approach of instrument-based audio subgroups that echoes his days working on large-format consoles has helped to keep things manageable as the channel count has risen. “If you start from scratch on every mix you do, it can be a bit of a nightmare,” admits Tayler. “Having an eight-channel sort of mentality means that I have subgroups ready for such things as drums, guitar parts, keyboards, backing vocals and main vocals. Now obviously you may have orchestral parts, percussion and other instruments, too, but having a clear signal path, effects, and routing does allow me to dive into a project fairly quickly.”
Different material inevitably calls for contrasting emphases or sonic enhancements, but in general, Tayler says his ethos is to “take what was happening in stereo a bit further. If you have a fairly [conventional] band line-up of guitars, bass, drums, keyboards and vocals, then you probably want it to feel as if the band is in front of you, but with sound that is going to envelop you. Nothing is set in stone, although personally I don’t like too much discrete isolation of things, especially [the practice of] taking the lead vocal and putting it purely in the centre speaker. That seems to be more of an idea developed for film to make sure that the dialogue has its own clear sort of space.”
Even a cursory glance of music audio forums confirms that listener opinions on the remixing of classic albums can be decidedly, well, mixed. There will be some music fans “who will want you to be strictly faithful to the arrangements, the sounds, the positioning and all the rest – just spread out more – whereas others will expect a remix to be more radical. So I think the task has to be to find the sort of sweet spot where you are respectful but also able to enhance the original.”
Like all of the contributors to this article, Tayler – who has also released a series of acclaimed albums under his own name, including 2021’s Da Capo – seems assured that the new era of immersive audio ushered in during the past 10 years is here to stay. With more content being issued in the new formats and home system price points gradually coming down, it also might not be too long before any lingering questions over mass-market accessibility begin to dissolve.
No wonder, then that Tayler is feeling enthusiastic about his journey into Atmos. “I am aware that some listeners expect [immersive mixes] to be gimmicky, with things spinning around, up and down,” he laughs. “But that’s not what I am going for. Instead, I like the idea of taking my existing philosophy into a more immersive space. It’s very exciting.”
1 Readers' comment