13.4 DSP Basics

Every sound a game makes — footsteps, gunfire, a character's voice, background music — ends up as the same thing by the time it reaches a speaker: a long list of numbers. This chapter is about that list. How sound becomes numbers in the first place, and the small pieces of math you use to change those numbers: turn them up, blend two sounds together, smooth out harshness, add an echo, or shape a note's volume over time. This whole field is called DSP (Digital Signal Processing — processing that treats a signal, here sound, as data: one number at a time, in a loop). None of it needs anything beyond what you already know: arrays, loops, a for over a buffer, and a little arithmetic.

We build it in the order a real audio engine would: what a sample even is, generating one from scratch, turning it up or down, mixing two together safely, filtering it, delaying it into an echo, thinking about it in terms of frequency instead of time, shaping it with an envelope, and finally, where all of this code actually runs inside a game — a strict, no-allocation callback on its own dedicated thread.

1. Sound as Data: Samples, Sample Rate, and Amplitude

Sound in the real world is air pressure wiggling up and down very fast. A microphone turns that wiggle into a voltage that also wiggles up and down smoothly, continuously, with no gaps. A computer cannot store something continuous — it can only store a fixed number of numbers. So to turn sound into data, the computer measures the height of that wiggle at regular, fixed moments in time. Each single measurement is called a sample: one number, usually a floating-point value roughly between -1.0 and +1.0. How high or low that number is at a given instant is the amplitude (loudness/pressure at that one moment — 0.0 means silence at that instant, +1.0 or -1.0 means as loud as the format allows).

How often those measurements are taken is the sample rate, measured in Hz (times per second). CD-quality audio and most game audio uses 44100 Hz (44,100 samples every second), sometimes written 44.1kHz. Some engines use 48000 Hz instead. Why so high? Human hearing tops out around 20,000 Hz, and a rule from information theory (the Nyquist theorem — we will not prove it here) says you need at least double the highest frequency you want to capture. 44100 is comfortably above double 20,000.

All those samples, one after another, are stored in an array — in audio code this array is almost always called a buffer. Plotting a buffer's values over time draws a picture called the waveform — the shape you see in any audio editor.

2. Making a Sound From Nothing: the Sine Wave

The simplest possible sound is a pure tone: a single, smooth sine wave. It is the building block everything else in this chapter is described in terms of (more on that in section 9). A sine wave has a frequency (how many full wiggles per second, in Hz — this is what we perceive as pitch) and an amplitude (how tall the wiggle is — this is what we perceive as loudness).

The math for one sample is: sample = amplitude * sin(2 * pi * frequency * n / sampleRate), where n is the sample's index (sample 0, sample 1, sample 2, ...). The part inside sin(...) is called the phase — the current position around the wave's circle, in radians. Instead of recomputing that whole expression from scratch for every sample, it is simpler and cheaper to keep a running phase value and just add a fixed step to it each time:

#include <cmath>

void FillSineBuffer(float* buffer, int numSamples, float frequency,
                     float sampleRate, float amplitude, float& phase)
{
    float phaseStep = 2.0f * 3.14159265f * frequency / sampleRate;
    for (int n = 0; n < numSamples; n++)
    {
        buffer[n] = amplitude * std::sin(phase);
        phase += phaseStep;
        if (phase > 2.0f * 3.14159265f)
            phase -= 2.0f * 3.14159265f;   // keep the angle from growing forever
    }
}

phase is passed by reference (float&) and kept outside the function on purpose: if you call FillSineBuffer again for the next chunk of audio, the wave has to keep picking up exactly where it left off, or you get an audible click at the seam between buffers. Let's trace 5 samples of a 440 Hz tone (the musical note A4) at 44100 Hz, amplitude 0.5, starting from phase = 0:

float phase = 0.0f;
float buf[5];
FillSineBuffer(buf, 5, 440.0f, 44100.0f, 0.5f, phase);
for (int i = 0; i < 5; i++)
    printf("%.4f\n", buf[i]);

Output:

0.0000
0.0313
0.0625
0.0935
0.1241

Each step forward adds the same fixed phaseStep (about 0.0627 radians here), so sin(phase) climbs smoothly from 0 upward — this is the very start of the wave's rise toward its peak. A higher frequency makes phaseStep bigger, so the sine climbs and repeats faster (more wiggles per second, higher pitch). A bigger amplitude just scales every sample up or down without changing the shape at all — which is exactly the next section's topic.

3. Volume Is Multiplication

Turning a sound up or down is nothing more than multiplying every sample by a number, called gain (a multiplier applied to a signal; when it is user-facing it is usually called volume). A gain of 1.0 leaves the sound unchanged, 0.5 makes it quieter, 0.0 makes it silent, and anything above 1.0 makes it louder than it originally was.

void ApplyVolume(float* buffer, int numSamples, float volume)
{
    for (int n = 0; n < numSamples; n++)
        buffer[n] *= volume;
}

float samples[4] = { 0.2f, -0.4f, 0.6f, -0.8f };
ApplyVolume(samples, 4, 0.5f);
for (int i = 0; i < 4; i++)
    printf("%.4f\n", samples[i]);

Output:

0.1000
-0.2000
0.3000
-0.4000

Every value shrank to exactly half of what it was, positive or negative — multiplying by 0.5 pulls every sample toward 0.0 equally. That is the entire idea behind a volume slider in a game's settings menu: read the slider's value (0.0 to 1.0) and multiply it into the buffer before the sound reaches the speaker.

4. Mixing Is Addition — and the Clipping Problem

When two sounds play at the same time — footsteps and background music, for example — the engine has to combine them into one buffer before it reaches the speaker. Mixing two buffers together is nothing more than adding their samples together, position by position.

float Clamp(float x, float lo, float hi)
{
    if (x < lo) return lo;
    if (x > hi) return hi;
    return x;
}

void MixBuffers(float* dst, const float* src, int numSamples)
{
    for (int n = 0; n < numSamples; n++)
    {
        float sum = dst[n] + src[n];
        dst[n] = Clamp(sum, -1.0f, 1.0f);   // see the warning below
    }
}

Trace it with a footsteps buffer (dry) and a sound effect buffer (sfx) that both happen to be loud at the same moment:

float dry[3] = { 0.6f, 0.7f, -0.5f };
float sfx[3] = { 0.5f, 0.6f, -0.6f };
MixBuffers(dry, sfx, 3);
for (int i = 0; i < 3; i++)
    printf("%.4f\n", dry[i]);

Output:

1.0000
1.0000
-1.0000

Before clamping, the raw sums were 1.1, 1.3, and -1.1 — every one of them outside the -1.0 .. +1.0 range a speaker (and the audio format itself) can represent. This is called clipping: any sample past the limit gets chopped flat instead of reaching its true peak.

5. Gain and Pan: One Sample, Two Channels

A mono (single-speaker) sound is one buffer, one number per moment in time. A stereo (left/right speaker) sound needs two numbers per moment — one for the left channel, one for the right. Game engines almost always store stereo audio interleaved: not two separate arrays, but one array alternating left, right, left, right, ....

Pan controls how much of a mono sound goes to the left speaker versus the right (a value from -1.0 full left, to 0.0 centered, to +1.0 full right). A simple (linear) pan just splits one sample into two differently-scaled copies:

void ApplyPan(float sampleIn, float pan, float& outLeft, float& outRight)
{
    // pan: -1.0 = full left, 0.0 = center, 1.0 = full right
    outLeft  = sampleIn * 0.5f * (1.0f - pan);
    outRight = sampleIn * 0.5f * (1.0f + pan);
}

float left, right;
ApplyPan(0.8f, -0.5f, left, right);
printf("left=%.4f right=%.4f\n", left, right);

Output:

left=0.6000 right=0.2000

With pan = -0.5 (leaning left), the left channel keeps most of the signal (0.6) and the right channel gets less (0.2). At pan = 0 both channels would get exactly half.

6. Filters, Part 1: the One-Pole Low-Pass Filter

A filter reshapes a signal based on how fast it is changing, not based on how loud it is. A low-pass filter lets slow, gradual changes through and smooths away fast, sudden ones — which in sound terms means it lets low pitches through and muffles high pitches (a "highs" cut, like a sound heard through a wall or underwater).

The simplest possible low-pass filter needs only one line of math, called a recurrence relation (a formula that defines the next value in terms of the previous one — you saw this idea with recursion in an earlier chapter, but here it runs forward through a buffer instead of calling itself):

Each new output is the old output nudged part of the way toward the new input, instead of jumping straight to it. That nudging is exactly what smooths out fast wiggles: a real one-pole filter needs to keep only one number of memory between calls (y[n-1]), which is why it is called "one-pole."

struct OnePoleLowPass
{
    float a;               // smoothing coefficient, 0..1
    float y_prev = 0.0f;   // filter memory: the last output

    float Process(float x)
    {
        y_prev = y_prev + a * (x - y_prev);
        return y_prev;
    }
};

Trace it against a sudden, sustained input — like a loud sound starting abruptly and staying loud (x = 1.0, over and over), with a = 0.3:

OnePoleLowPass lp;
lp.a = 0.3f;
for (int i = 0; i < 5; i++)
    printf("%.3f\n", lp.Process(1.0f));

Output:

0.300
0.510
0.657
0.760
0.832

Instead of jumping to 1.000 instantly, the filter eases toward it: each step closes 30% of the remaining gap (a = 0.3). A sudden jump is a fast change — exactly the kind of thing a low-pass filter is built to smooth away. Feed it a signal that is already slow and steady and it barely changes it at all.

7. Filters, Part 2: High-Pass and Band-Pass

High-pass

A high-pass filter does the opposite of a low-pass: it lets fast changes (high pitches) through and blocks slow, steady ones (low pitches and rumble). The cheapest way to build one is to reuse the low-pass filter you already have: whatever the low-pass smoothed away and kept out is the high-pass result. Subtract the low-pass output from the original input, and what's left is everything the low-pass filter removed.

struct OnePoleHighPass
{
    OnePoleLowPass lowpass;   // reuse the low-pass from section 6

    float Process(float x)
    {
        float low = lowpass.Process(x);
        return x - low;        // whatever the low-pass did NOT keep
    }
};

Run the same sustained x = 1.0 input from section 6 through it (a = 0.3):

0.700
0.490
0.343
0.240
0.168

These are exactly 1.0 minus each low-pass value from section 6 (1.0 - 0.300 = 0.700, and so on). The high-pass output starts strong (the sudden jump itself is a big, fast change) and fades toward zero as the signal settles into something steady — which makes sense: a high-pass filter only reacts to change, and a constant signal eventually has none left to react to.

Band-pass

A band-pass filter passes only a middle range ("band") of frequencies, blocking both the lows and the highs outside it — the classic "telephone voice" or "old radio" sound. Build one by chaining a high-pass (to remove the lows) into a low-pass (to remove the highs), so only what survives both stages gets through:

struct OnePoleBandPass
{
    OnePoleHighPass highpass;
    OnePoleLowPass  lowpass;

    float Process(float x)
    {
        float noLows = highpass.Process(x);
        return lowpass.Process(noLows);
    }
};

Feed the same sustained step (x = 1.0, repeated) into OnePoleBandPass and the low, steady part gets blocked by the internal high-pass first, then whatever quick energy is left gets smoothed further by the internal low-pass. The result peaks briefly right after the jump and fades back toward zero — it passes neither a pure steady tone nor the very sharpest instant of change, only what sits in between.

8. Delay and Echo: a Buffer That Remembers the Past

An echo repeats a sound after a short delay, quieter each time. Building one needs a way to remember samples from the past — a delay line: a fixed-size buffer used as a ring (a circular buffer, wrapping back to index 0 after the last slot). New samples get written in; old samples, sitting a fixed number of slots behind, get read back out.

struct Delay
{
    static const int kSize = 4;    // kept tiny so it is easy to trace by hand
    float buffer[kSize] = { 0.0f, 0.0f, 0.0f, 0.0f };
    int writePos = 0;
    float feedback = 0.5f;         // how much of the echo repeats again -- keep < 1.0

    float Process(float input)
    {
        float wet = buffer[writePos];                // the echo: written here N samples ago
        buffer[writePos] = input + wet * feedback;    // new input + a fraction of the echo
        writePos = (writePos + 1) % kSize;            // move forward, wrap after the last slot
        return wet;                                   // caller mixes this with the dry signal
    }
};

Process only hands back the echoed ("wet") sample — combining it with the original ("dry") sample is just the mixing (addition) from section 4:

Delay delay;
float dry[9] = { 1.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f };
for (int n = 0; n < 9; n++)
{
    float wet = delay.Process(dry[n]);
    float outSample = dry[n] + wet * 0.5f;   // 50% wet mix under the dry signal
    printf("n=%d  output=%.4f\n", n, outSample);
}

Output (only the interesting samples shown; the rest are 0.0000):

n=0  output=1.0000
n=4  output=0.5000
n=8  output=0.2500

A single loud "clap" at n=0 comes back as an echo 4 samples later (n=4), scaled down by the 0.5 wet mix. Four samples after that (n=8), it comes back again — this time also scaled by feedback = 0.5 on top, so it is half as loud as the first echo. Each additional lap around the buffer multiplies by feedback again, which is exactly why real echoes fade out instead of repeating forever.

A basic reverb (the sense of a sound in a room, not just a single repeat) is built from the same idea, scaled up: instead of one delay line, use several delay lines of different, non-matching lengths, feeding into and mixing with each other. So many overlapping, decaying echoes arrive so close together that your ear stops hearing individual repeats and instead hears one smooth, spacious tail — which is exactly what a room full of reflecting surfaces does to a sound in real life.

9. The Frequency Domain and the FFT

Section 2 built a single pure sine wave. A real sound — a voice, a guitar string, an explosion — is not one sine wave; it is many sine waves of different frequencies and volumes, all added together at once. This is the central idea of the frequency domain: any repeating signal can be described as a sum of simple sine waves, each with its own frequency and amplitude. Describing a sound this way (as a list of "how much of each frequency") is just as valid as describing it as a buffer of samples over time (the time domain, which is everything we have done so far in this chapter) — they are two views of the exact same sound.

Building a "richer" tone really is just addition, the same operation as mixing in section 4 — only here we are adding two different pitches of the same instrument instead of two different sounds:

const int N = 8;
float toneA[N];
float toneB[N];
float complexTone[N];

float phaseA = 0.0f;
float phaseB = 0.0f;
FillSineBuffer(toneA, N, 220.0f, 44100.0f, 0.3f,  phaseA);  // the fundamental note
FillSineBuffer(toneB, N, 440.0f, 44100.0f, 0.15f, phaseB);  // one octave up, quieter

for (int n = 0; n < N; n++)
    complexTone[n] = toneA[n] + toneB[n];   // this line IS "mixing" from section 4

At n = 0, both phases start at 0, so complexTone[0] = 0.3 * sin(0) + 0.15 * sin(0) = 0.0000. From there, the two sine waves climb at different speeds (220 Hz and 440 Hz have different phaseStep values), so the sum stops looking like one smooth sine and starts looking bumpy — that bumpiness is literally two frequencies overlapping in the same numbers.

Going the other direction — taking a buffer of samples and figuring out which frequencies (and how much of each) are hiding inside it — is what the FFT (Fast Fourier Transform, an algorithm; we will not implement one here) does. Feed it a chunk of time-domain samples, and it hands back a set of frequency bins, each one telling you how much energy is present around that frequency.

An EQ (equalizer — the "bass/mid/treble" style control in an audio app) is exactly that last sentence: boost or cut specific frequency ranges. Conceptually, that means running the FFT, multiplying certain bins by a gain, then running the inverse operation to get back to a time-domain buffer:

// Pseudocode -- shows the IDEA of an FFT-based EQ; not runnable as-is.
float freqBins[bins] = FFT(timeBuffer);       // time domain -> frequency domain

freqBins[bassRange]   *= 1.5f;                 // boost the bass frequencies
freqBins[trebleRange] *= 0.6f;                 // cut the treble frequencies

float timeBuffer2[N] = InverseFFT(freqBins);   // frequency domain -> time domain again

10. Envelopes: Shaping a Sound Over Time (ADSR)

Section 3 multiplied every sample by one constant volume. An envelope is the same idea stretched across time: instead of one fixed multiplier, it is a multiplier that changes over the life of a sound — quiet at first, rising to a peak, settling down, and fading out. The most common shape, used by almost every synthesizer and sampler, is ADSR: Attack, Decay, Sustain, Release.

enum class EnvelopeState { Attack, Decay, Sustain, Release, Idle };

struct ADSR
{
    float attackStep;    // added to value each sample during Attack
    float decayStep;     // subtracted from value each sample during Decay
    float sustainLevel;  // held level during Sustain
    float releaseStep;   // subtracted from value each sample during Release
    float value = 0.0f;
    EnvelopeState state = EnvelopeState::Attack;

    void NoteOff() { state = EnvelopeState::Release; }

    float Next()
    {
        switch (state)
        {
            case EnvelopeState::Attack:
                value += attackStep;
                if (value >= 1.0f) { value = 1.0f; state = EnvelopeState::Decay; }
                break;
            case EnvelopeState::Decay:
                value -= decayStep;
                if (value <= sustainLevel) { value = sustainLevel; state = EnvelopeState::Sustain; }
                break;
            case EnvelopeState::Sustain:
                break;   // holds until NoteOff() is called
            case EnvelopeState::Release:
                value -= releaseStep;
                if (value <= 0.0f) { value = 0.0f; state = EnvelopeState::Idle; }
                break;
            case EnvelopeState::Idle:
                break;
        }
        return value;
    }
};

Trace it with attackStep = 1/3, decayStep = 0.25, sustainLevel = 0.5, releaseStep = 0.25, calling Next() once per sample and calling NoteOff() right after the 6th call:

call 1 (Attack):   0.3333
call 2 (Attack):   0.6667
call 3 (Attack):   1.0000   -> reaches peak, switches to Decay
call 4 (Decay):    0.7500
call 5 (Decay):    0.5000   -> reaches sustainLevel, switches to Sustain
call 6 (Sustain):  0.5000   -> NoteOff() called here
call 7 (Release):  0.2500
call 8 (Release):  0.0000   -> reaches 0, switches to Idle

Multiply this returned value into a sound's samples every sample (exactly like ApplyVolume in section 3, except the multiplier itself now comes from Next() instead of being a fixed number) and you get a note that fades in, settles, holds while a key or trigger is pressed, and fades out cleanly when released — instead of an abrupt on/off click.

11. Where DSP Plugs Into an Engine: the Audio Callback

Every piece of code in this chapter has one thing in common: it fills or modifies a buffer of samples, a chunk at a time. That is exactly how a real engine asks for audio. On its own dedicated audio thread (separate from the game thread that runs Update(), physics, and AI), the engine periodically calls into your code through an audio callback — a function whose whole job is: "fill this buffer of samples, right now, before it's due to reach the speaker."

Because the deadline is strict and the consequence of missing it is instantly audible, code that runs on the audio thread follows one hard rule that nothing else in this book has needed yet: do not allocate memory there — no new, no malloc, no resizing a List or std::vector, no locking a mutex the game thread might be holding, no logging, no file access. Any of those can pause the audio thread for an unpredictable moment (a garbage collector run, an OS memory request, a lock waiting on another thread) — and an unpredictable pause on this thread is exactly what causes the click or pop you sometimes hear in a poorly-optimized game. Every buffer and every filter's state should be allocated once, ahead of time, and only read from or written to inside the callback itself.

Unity: OnAudioFilterRead (C#)

Unity calls OnAudioFilterRead on the audio thread for any AudioSource on the same GameObject. It hands you the buffer directly — data is interleaved exactly like section 5's diagram — so you filter it in place, with no allocation at all:

using UnityEngine;

public class SimpleLowPassFilter : MonoBehaviour
{
    public float cutoffAmount = 0.3f;     // 0..1, smaller = more filtering

    // Filter memory lives here, as fields -- allocated once, never inside the callback.
    private float lowpassStateL = 0.0f;
    private float lowpassStateR = 0.0f;

    // Unity calls this on the AUDIO THREAD, not the main thread.
    // It must finish fast, and it must not allocate.
    void OnAudioFilterRead(float[] data, int channels)
    {
        for (int i = 0; i < data.Length; i += channels)
        {
            lowpassStateL = lowpassStateL + cutoffAmount * (data[i] - lowpassStateL);
            data[i] = lowpassStateL;

            if (channels > 1)
            {
                lowpassStateR = lowpassStateR + cutoffAmount * (data[i + 1] - lowpassStateR);
                data[i + 1] = lowpassStateR;
            }
        }
    }
}

This is exactly the one-pole low-pass from section 6, applied sample by sample to a real Unity buffer — lowpassStateL/lowpassStateR play the role of y_prev, and because they are fields on the object (not local variables), they keep their value from one callback to the next, the same way phase had to in section 2.

A native engine callback (C++)

Lower-level engines and native audio APIs (the style used by libraries like miniaudio, PortAudio, WASAPI, or CoreAudio) hand you a raw pointer and a frame count instead of a managed array, but the shape of the job is identical:

struct DspState
{
    OnePoleLowPass filterL;
    OnePoleLowPass filterR;
};   // allocated ONCE, e.g. in an Init() function, long before playback starts

void AudioCallback(float* outputBuffer, int numFrames, int numChannels, void* userData)
{
    DspState* dsp = static_cast<DspState*>(userData);   // NOT allocated here

    for (int frame = 0; frame < numFrames; frame++)
    {
        int i = frame * numChannels;
        outputBuffer[i] = dsp->filterL.Process(outputBuffer[i]);
        if (numChannels > 1)
            outputBuffer[i + 1] = dsp->filterR.Process(outputBuffer[i + 1]);
    }
}

12. Glossary

• Sample — one number representing a sound's amplitude at one instant in time.
• Sample rate — how many samples are taken per second, in Hz (e.g. 44100 Hz / 44.1kHz).
• Amplitude — how loud a signal is at one instant; typically stored as a float roughly between -1.0 and +1.0.
• Buffer — an array of samples, the basic unit that audio code reads and writes.
• Waveform — the shape you get plotting a buffer's samples over time.
• Gain / Volume — a multiplier applied to a signal's samples to make it louder or quieter.
• Clipping — the distortion that happens when a sample's value is chopped flat because it went past what the format can represent (usually past -1.0 or +1.0).
• Mixing — combining two or more sounds into one by adding their samples together, position by position.
• Pan — how much of a signal goes to the left channel versus the right.
• Filter — DSP that reshapes a signal based on how fast it changes (its frequency content), such as low-pass, high-pass, or band-pass.
• Recurrence relation — a formula defining the next value from the previous one(s); the basis of the one-pole filter's y[n] = y[n-1] + a * (x[n] - y[n-1]).
• Cutoff frequency — the pitch a filter is centered around; the point where it starts letting less signal through.
• Delay line — a fixed-size circular buffer used to remember past samples so they can be replayed later, as an echo.
• Feedback — the fraction of a delay's output that gets written back into the delay buffer, controlling how many times an echo repeats and how fast it fades.
• Reverb — a dense collection of overlapping, decaying echoes that gives the sense of a sound existing inside a physical space.
• Frequency domain / Time domain — two ways of describing the same signal: as a sum of frequencies and their strengths, or as a buffer of amplitude values over time.
• FFT (Fast Fourier Transform) — an algorithm that converts a time-domain buffer into its frequency-domain representation (a set of frequency bins).
• EQ (Equalizer) — an effect that boosts or cuts specific frequency ranges of a signal.
• Envelope / ADSR — a time-varying multiplier (Attack, Decay, Sustain, Release) used to shape how a sound's volume (or another parameter) changes over its lifetime.
• Audio callback — the function an engine or audio API calls to request the next chunk of samples to play.
• Audio thread — the dedicated thread the audio callback runs on, with a strict real-time deadline; must never allocate, lock, or block.

13. Exercises

public class BadFilter : MonoBehaviour
{
    public float cutoffAmount = 0.3f;

    void OnAudioFilterRead(float[] data, int channels)
    {
        float[] smoothed = new float[data.Length];   // temp buffer
        float state = 0.0f;
        for (int i = 0; i < data.Length; i++)
        {
            state = state + cutoffAmount * (data[i] - state);
            smoothed[i] = state;
        }
        for (int i = 0; i < data.Length; i++)
            data[i] = smoothed[i];
    }
}

That covers the core of game audio DSP: sound as a buffer of samples at a fixed sample rate, volume as multiplication, mixing as addition (watching for clipping), splitting a signal across channels with gain and pan, shaping frequency content with one-pole filters, remembering the past with a delay line to build an echo, thinking about a signal in the frequency domain instead of the time domain, shaping loudness over time with an envelope, and finally, the strict no-allocation rules of the audio thread where all of this code actually has to run. Every piece of it reduces to the same handful of operations — multiply, add, remember one previous value — repeated over a buffer, over and over, many times a second.