The function of sound in games

The use of interactive sound in video games has largely been a work of people working independently, reinventing the wheel on a case-by-case basis. Very little thought has been put into defining a framework general enough to be used in a wide range of applications. Communities of music composers and programmers can informally address this problem and conferences are helping facilitate that by featuring audio and music tracks, but innovations will come faster if there is a way to analyze why certain techniques work where others fail.

This hasn’t happened yet for video games partly because of the nature of the industry. Working on single project after single project often leads designers to be shortsighted and work on solving the problem in front of them, drawing only slightly upon previous works and works of their peers. It’s also partly because video game music (Nobuo Uematsu fans excluded) often doesn’t get the attention it deserves amongst media and consumers. Sound is very important for drawing people into a game, doing it in such subtle ways that people don’t realize the effect is has on them. In fact, many times when you do notice the music it’s because something is off. Mahito Yokota (audio designer of Super Mario Galaxy) says that “players are able to focus better on the game play” when the game tempo and music tempos are synchronized in a meaningful way (http://us.wii.com/iwata_asks_vol3_index.jsp).

So what would such a framework look like?
Studying music, I spent most of the time determining how notes, chords and counterpoint functioned in a larger piece of music. Most time in composition class was spent justifying musical choices to my teacher. Nothing was really disapproved of as long as it had a functional reason for existing. This approach can obviously be taken too far, become overly academic and lead to terrible sounding music with highly detailed internal rules, but for it’s purpose – to teach rather than to create – it works great. It teaches the student to understand how each note functions within the context of the whole piece. Although you may not explicitly use this knowledge when composing, it can still inform your choices, give you places to start and allow you to know why something just doesn’t sound right.

And it doesn’t take a huge step of faith to see how this can be applied to music in games: all we need to do is find a way to analyze how sound functions in game space. Luckily there’s a lot of overlap in how both games and music unfold over time.