Can Realistic Audio Enhance Game Play?

The desire to enhance the gaming experience and gamer performance has been the engine driving continual innovations in GPUs, devices such as gaming consoles and HMDs, and software such as Unity and Unreal.  While a majority of the spotlight has been on graphics, realistic audio has emerged as a critical component in the gamer’s toolbox.  Its need is especially acute in XR environments, which have become more popular due to the success of VR headsets like the Oculus Quest 2. VR environments up the ante in demands for realism, as these environments want users to believe that they are in another world. If the audio is not realistic, this will hamper the illusion, degrading the user experience. Even in AR in order to believe a virtual object really exists in your living room, sound plays a key role.

Can Realistic Audio Enhance Gameplay?

Realistic audio has many benefits, including improving the sense of presence, entertainment, navigation, situational awareness and motor performance.  It is not hard to see how this type of audio can help you in a game, like having better aim when you are shooting a laser at an oncoming missile, or preventing you from being surprised by an attacker stealthily approaching from behind.  In the missile scenario, the missile’s sound helps you to zero in quickly on the most effective shot since you have both visual and sound ques, while in the sneaky attacker scenario, realistic audio alerts you to the existence and location of a threat before you even see it.  Realistic audio can also help in multiplayer communication scenarios, allowing the player to more quickly process who is speaking and what they are saying.

There have been several research studies that attempt to quantify the benefits of simulating realistic sound by looking at its effects on the sense of presence or task performance.  Fewer studies look at actual gaming performance, but it is not hard to appreciate that improved immersion and better task execution abilities can lead to higher scores.

Presence, that sense of “being there”, is usually measured by exposing subjects to experiences with or without some of the feature set of realistic audio and having them complete a standardized survey tool.  As you would expect, realistic audio does indeed lead to an increased sense of presence, and this can happen in XR environments, or specifically created laboratory settings (see 1,2,3,4).  Not only can this help with gaming performance, but also improves engagement and an emotional connection, which can be seen when measuring specific physiologic parameters (2,3).

Common methods of exploring how realistic audio can improve task performance include  measuring how accurate the subjects can be in a navigational task (5) or how quickly subjects can identify an object that appears somewhere in the space around them (6).  As expected, when spatialized audio was used, subject performance was much improved compared to when the subjects tried to perform the task without spatialization.  Such tasks can be more generalizable to improved aiming or moving more quickly through a map to add to the competitiveness of a player.

Fewer studies are available on specific game genre performance in XR, but as the hardware and software become less expensive and more accessible, these will increase.  One study looked at threat level perception in a FPS scenario using different realistic audio feature sets, and found specific realistic audio features are effective depending on the type of environment (forest, home, industrial) (7). 

Realistic audio is critical to an engaging and emotionally captivating game, and with the resultant increased attention by the player, their gaming performance will be enhanced.  Add to this the benefits of improved task performance afforded by realistic audio, and a gamer has a highly effective modality in which to give them that competitive edge.

  1. N. Khenak, J. Vézien, D. Théry and P. Bourdot, "Spatial Presence in Real and Remote Immersive Environments and the Effect of Multisensory Stimulation," in Presence, vol. 27, no. 3, pp. 287-308, July 2020, doi: 10.1162/pres_a_00332.
  2. Kobayashi, Maori & Ueno, Kanako & Ise, Shiro. (2015). The Effects of Spatialized Sounds on the Sense of Presence in Auditory Virtual Environments: A Psychological and Physiological Study. Presence: Teleoperators and Virtual Environments. 24. 163-174. 10.1162/PRES_a_00226.
  3. https://projekter.aau.dk/projekter/en/studentthesis/effekten-af-spatial-lyd-paa-indlevelse-tilstedevaerelse-og-fysiologisk-respons-i-computer-spil(9bc49b5e-0e7c-4624-8297-60db533764ea).html  
  4. Larsson, Pontus & Väljamäe, Aleksander & Västfjäll, Daniel & Tajadura-Jiménez, Ana & Kleiner, Mendel. (2010). Auditory-Induced Presence in Mixed Reality Environments and Related Technology. 10.1007/978-1-84882-733-2_8.  
  5. Rumiński, D. An experimental study of spatial sound usefulness in searching and navigating through AR environments. Virtual Reality 19, 223–233 (2015). https://doi.org/10.1007/s10055-015-0274-4
  6. A. N. Moraes, R. Flynn, A. Hines and N. Murray, "Evaluating the User in a Sound Localisation Task in a Virtual Reality Application," 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), Athlone, Ireland, 2020, pp. 1-6, doi: 10.1109/QoMEX48832.2020.9123136.
  1. Konstantin Semionov and Iain McGregor. 2020. Effect of various spatial auditory cues on the perception of threat in a first-person shooter video game. In Proceedings of the 15th International Conference on Audio Mostly (AM '20). Association for Computing Machinery, New York, NY, USA, 22–29. DOI:https://doi.org/10.1145/3411109.3411119
Back to Recent Blogs