What will the Metaverse sound like?

A sound designer’s opinion on the kind of audio experiences we should thrive for in the future.

We’re not there yet

The other day I was having a conversation with a friend regarding everything that is going on around Web 3.0. At some point he asked me, « to you as a sound designer, what do you think the Metaverse will sound like? » The first thing that came to mind is « well, it will probably sound realistic enough but it will probably never sound right! ». Not because the technology is not ready or because we have not been able to create successful audiovisual experiences (have you played Cyberpunk 2077?), but because a lot of studios and production companies still have not realized the full potential of well-balanced, properly-mixed and truly immersive audio. 

Audio matters

Having worked as a sound designer for many immersive experiences – from projection mapping and AR (Augmented Reality) to full-on VR (Virtual Reality) video games – I know for a fact that good audio can very often make the difference between a successful and an OK experience. It has been proven countless times that high-quality audio is more important that high-quality images (just look for yourself). However, a lot of production companies still forget to allocate the right resources to sound and music. For example, a study by Psychologists Eryn J. Newman and Norbert Schwarz proved essentially that, bad audio makes people seem less clever and less likable. Everybody has an opinion about the Metaverse and what it means for the future of humanity but I think we can all agree that nobody wants their avatar to be regarded as dumb!

The hierarchy of sound

There is so many layers to the way we perceive sound and it is likely that if we want to enjoy a digitally-created world, we need to figure out a way to unify the hierarchy of audio informations. Our brain and ears are designed to put the most important first and very often a loud noise or someone screaming is the first sign that we may be in danger. Likewise, the ability to clearly perceive the voices and sounds around give us hints about our place in a society. Finally let’s not forget about the immense power of music and how it affects our mood and emotions everyday. It seems apparent that we need to make sure that all these audio sources are put in context and adapted to each individual’s preferences and tolerances.

Limitless creativity

Who says we can’t invent a part of the Metaverse that relies solely on carefully crafted audio experiences so that we can relax our eyes and let our mind navigate in a sea of beautiful musical ambiences? This is where it gets interesting for me as a sound designer. We get to invent a world where we are not limited by our natural abilities and where we can dedicate time to enhance our natural abilities in a playful and meaningful way. What if we could feel what it means to be a giant blue whale in the ocean just with sound? What if you could develop synesthesia and learn music theory faster? I personally see tremendous potential for exploring the uncharted parts of our senses and developing new abilities, but for that the technology needs to be ready.

Technology

The good news is that the technology for creating immersive audio is already here in most parts. Back in 2019, our studio took part in the development of a VR experience that was going to air on commercial flights. The obvious constraint is that the player is sitting down throughout the experience so their head can only move so much. By programming an Ambisonics mix, we were able to give the illusions that the objects were moving around in this space. For example, if an object is coming from the right, you perceive its sound on the right ear, but as soon as you look at it coming towards you, your auditory perception changes and now the sound comes to you through both ears. That technology is not new and it is still perfectible but it’s a solid start. Furthermore, video game developers and sound designers can now easily create convincing and truly immersive experiences by using audio tools like FMod or Wwise that attach directly to the game engine and instantly react to the player’s environment. Finally, huge companies like Apple are investing in 360 audio and Dolby Atmos technologies. These tools give us affordable hardware that can decode and understand the relative complexity of audio immersive experiences. 

Accessibility and health

An important part of creating the Metaverse to me is creating the most inclusive environment possible and allowing all of us humans to transcend our limitations. People who cannot fully partially perceive images or sounds in the real world need to be able to navigate the Metaverse easily and whether it is through special equipment or through native design, we need to take that into account. I am saying that not only because of my desire for inclusivity and solidarity, but also because those of us who can hear well, have a lot to learn from people with hearing disabilities. Furthermore, it is known for centuries that sound has healing properties and there is still a lot to learn in this field.

What space for brands?

Part of our job at Sound Object is to help brands push through the noise and stand out with a meaningful and tailored audio identity. A lot of brands are already investing in Web 3.0 and if they want to survive without pushing unwanted ads into people’s face all day long, their brand identity needs to be clear and instantly recognizable. If you ask us, sound is the perfect tool for that! However it, again, raises the question of audio hierarchy. No one wants to be overwhelmed by bright jingles to remind them to buy the next NFT sneakers or explore the next beer-branded theme park. Yet, brands deserve their space in the Metaverse and we need to carefully think about how they will interact with the consumers before this idealized world becomes just a giant version of Times Square!

So, will the Metaverse ever sound « right »?

I do not pretend to know what experiencing the Metaverse will be like but I am pretty sure that it has to become a place where humans can experience a superior level of perception and evolve in an environment that is stimulating rather than overwhelming. 

The best AAA video game studios and film directors have known for years that they need to make sure that the composers and sound designers are here at the very early stages of production. Considering the complexity behind creating any kind of Metaverse and even though it probably will not be the experience that we all want just yet, it is time we make sure it sounds great.