3.3 Concurrency & Multithreading

So far your C++ programs have done one thing at a time: run one line, then the next, in one long path from main to the end. That single path is called a thread. This chapter is about running more than one of these paths at once, which is how a real game engine uses all the processing power a modern computer has instead of leaving most of it idle.

Every section below follows the same shape you already know: a small runnable example, its real output, then a plain explanation of what happened. Threads add one new twist worth calling out up front: some of the programs in this chapter print a different result every time you run them, on purpose, because that unpredictability is exactly the problem this chapter teaches you to solve.

1. What a thread is, and why a game wants several

A thread is a single sequence of instructions that the CPU runs one after another. Every C++ program starts with exactly one thread — it begins at main and runs downward, line by line, exactly like every program you have written so far. A process (your running program) can create more threads, and each new thread runs its own sequence of instructions independently, at the same time as the others.

A CPU core is the physical piece of hardware that actually executes a thread's instructions. A modern gaming PC, console, or phone rarely has just one core — 6, 8, or 16 cores is normal today. If your whole game runs on a single thread, it can only ever use one of those cores. The other 5, 7, or 15 cores sit there doing nothing while your one thread struggles through everything the game needs done that frame.

And a frame is a lot of separate work: step the physics simulation, build the list of draw commands for the GPU, mix and stream audio, run AI decisions for every enemy, stream new assets in from disk. Much of that work does not depend on the other parts moment-to-moment, so it can run in parallel (truly at the same time, on different cores) instead of one after another.

That is the whole motivation for this chapter: more threads, used correctly, means more of the CPU is actually doing useful work at the same moment, which means a shorter frame time. "Used correctly" is doing a lot of work in that sentence — the rest of this chapter is about the mistakes that happen when threads are used carelessly, and the tools C++ gives you to avoid them.

2. Starting a thread: std::thread

C++'s standard library gives you std::thread (from the <thread> header). You give it a function, and it starts running that function on a new thread immediately — the constructor does not wait.

#include <thread>
#include <iostream>

void sayHello() {
    std::cout << "hello from a worker thread\n";
}

int main() {
    std::cout << "main thread starts\n";
    std::thread t(sayHello);        // starts running sayHello() right now, on a new thread
    std::cout << "main thread keeps going\n";
    t.join();                       // wait here until t finishes
    std::cout << "main thread done\n";
}

Output (one possible run — the middle two lines can swap):

main thread starts
main thread keeps going
hello from a worker thread
main thread done

Two lines are running truly independently: main printing "main thread keeps going", and the new thread printing "hello from a worker thread". Which one the operating system actually lets run first is not guaranteed — run this program several times and you may sometimes see "hello from a worker thread" printed before "main thread keeps going". That is not a bug; it means the two threads really did run concurrently, and the OS scheduler picked an order.

t.join() makes the calling thread (here, main) pause and wait until t finishes. Without a join (or its alternative, t.detach(), which lets the thread run fully independently and never needs to be waited on), the program has no defined way to know the thread is done.

3. The core danger: a data race

Here is the problem that makes multithreading hard. A data race happens when two or more threads read and write the same variable at the same time, with no coordination between them, and at least one of those accesses is a write. Let's see one happen.

#include <thread>
#include <iostream>

int counter = 0;   // shared by every thread that touches it -- that is the danger

void addOneMillion() {
    for (int i = 0; i < 1000000; i++) {
        counter++;   // looks like one step. it is NOT.
    }
}

int main() {
    std::thread t1(addOneMillion);
    std::thread t2(addOneMillion);
    t1.join();
    t2.join();
    std::cout << "counter = " << counter << "\n";
}

Two threads each add 1 to counter a million times, so we expect counter to end at exactly 2,000,000. Real output, though:

counter = 1417253

Wrong — and worse, not consistently wrong. Run this program again and you might get 1583920, or 1998447, or (rarely) the correct 2000000. The number is different nearly every run, on every machine. That unpredictability is the signature of a data race.

The reason: counter++ looks like a single step in the source code, but the CPU actually does it in three separate steps — load the current value into a register, add 1 to it, then store the result back to memory. When two threads run those three steps without any coordination, their steps can interleave in a bad order:

Multiply that lost update by a million iterations on two threads racing each other, and you get a final number well short of 2,000,000 — and a different shortfall every time you run it, because the exact interleaving depends on OS scheduling, which is never exactly the same twice.

4. Why this is undefined behavior, not just "a wrong number"

You have already met undefined behavior (UB) — code the C++ standard places no guarantees on at all, like reading past the end of an array or using a pointer after delete. A data race is UB too. The standard says: if two threads access the same memory without synchronization and at least one is a write, the behavior of the program is not defined.

That matters for a subtle reason beyond "you might get a wrong number." Because the standard guarantees nothing, the compiler is allowed to assume your program has no data races when it optimizes your code. It might reorder instructions, keep a value cached in a register instead of re-reading memory, or make other changes that are perfectly safe for race-free code but produce genuinely bizarre results — not just a wrong count, but infinite loops, crashes, or values that were never written by any thread — when a race is actually present. A data race is not "probably fine, just slightly off." Treat it exactly like the other UB you already know to avoid: fix it, do not tolerate it.

5. Fixing it: std::mutex

A mutex (short for "mutual exclusion", from <mutex>) is a lock that only one thread can hold at a time. Call m.lock() to acquire it; if another thread already holds it, your thread waits until it is free. Call m.unlock() to release it. The code between lock() and unlock() is called a critical section — a stretch of code guaranteed to run in only one thread at a time.

#include <thread>
#include <mutex>
#include <iostream>

int counter = 0;
std::mutex m;

void addOneMillion() {
    for (int i = 0; i < 1000000; i++) {
        m.lock();
        counter++;      // only one thread can be inside here at a time
        m.unlock();
    }
}

int main() {
    std::thread t1(addOneMillion);
    std::thread t2(addOneMillion);
    t1.join();
    t2.join();
    std::cout << "counter = " << counter << "\n";
}

Output — every single time you run it:

counter = 2000000

The mutex forces the three steps (load, add, store) to finish completely for one thread before the other thread is allowed to start its own load. No interleaving, no lost updates, always the correct total. The cost: a mutex is not free. Locking and unlocking, especially when many threads are fighting over the same one, can involve the operating system putting a thread to sleep and waking it later, which is far slower than a plain addition. Correctness comes first — you can worry about that cost once the program is actually right.

6. std::lock_guard: never forget to unlock

Calling lock() and unlock() by hand is risky. If an exception is thrown between them, or a bug adds an early return inside the critical section, unlock() never runs — the mutex stays locked forever, and every other thread that later calls lock() on it waits forever too. That is a real and common bug.

std::lock_guard fixes this the same way C++ always fixes "did you clean this up" problems: a small object whose constructor does the setup and whose destructor does the cleanup, automatically, no matter how the surrounding scope is exited. Its constructor calls lock(); when it goes out of scope — normal exit, early return, or an exception — its destructor calls unlock() for you.

void addOneMillion() {
    for (int i = 0; i < 1000000; i++) {
        std::lock_guard<std::mutex> guard(m);   // locks m right now
        counter++;
    }   // guard is destroyed here -> m.unlock() happens automatically
}

The output is identical to section 5 — still counter = 2000000 every time — but this version cannot leak a lock. Prefer std::lock_guard (or std::scoped_lock, which section 8 covers) over calling lock()/unlock() by hand in essentially all real code.

7. Atomics: std::atomic for a lock-free counter

For a simple case like a single counter, C++ offers a lighter tool: std::atomic<T> (from <atomic>). Operations on an atomic variable — increment, add, read, write — are guaranteed to happen as one indivisible (uninterruptible) step. No other thread can ever see it "half done." No lock(), no unlock(), no mutex object at all.

#include <thread>
#include <atomic>
#include <iostream>

std::atomic<int> counter{0};

void addOneMillion() {
    for (int i = 0; i < 1000000; i++) {
        counter++;   // atomic increment -- safe with no mutex
    }
}

int main() {
    std::thread t1(addOneMillion);
    std::thread t2(addOneMillion);
    t1.join();
    t2.join();
    std::cout << "counter = " << counter << "\n";
}

Output — correct every time:

counter = 2000000

Under the hood, std::atomic uses special CPU instructions (broadly known as compare-and-swap style instructions) that the hardware itself guarantees cannot be interrupted halfway. For a single counter or flag, this is usually noticeably faster than a mutex, because it never needs the operating system to put a thread to sleep.

Atomics only protect one variable per operation, though. If you need to update two related variables together and keep them consistent with each other — say, a position's x and y that must never be read as "half moved" — making each one an atomic separately does not help; another thread could still read a new x paired with an old y. That situation needs a mutex around both, so the whole update is one critical section. Rule of thumb: atomic for a single simple value; mutex for anything that touches more than one variable, or more than one step, that must stay consistent together.

8. Deadlock: two locks, the wrong order

Mutexes solve data races, but they introduce a new failure mode of their own: deadlock. Deadlock is when two (or more) threads each hold a lock the other one needs, and both wait forever. No crash, no error message — the program simply hangs.

#include <thread>
#include <mutex>

std::mutex mA, mB;

void threadFunc1() {
    std::lock_guard<std::mutex> lockA(mA);   // grabs mA first
    std::lock_guard<std::mutex> lockB(mB);   // then wants mB
    // ... use both resources ...
}

void threadFunc2() {
    std::lock_guard<std::mutex> lockB(mB);   // grabs mB first
    std::lock_guard<std::mutex> lockA(mA);   // then wants mA
    // ... use both resources ...
}

This program has no output to show — it can simply freeze forever. Here is a worked trace of the unlucky interleaving that causes it:

Thread 1 will not release mA until it gets mB. Thread 2 will not release mB until it gets mA. Neither ever happens. This is called a circular wait, and it is the classic cause of deadlock: two threads locking the same two mutexes in opposite order.

Two ways to fix it. First, the simple discipline: always lock mutexes in the same order everywhere in the codebase — for example, always mA before mB, never the reverse, no matter which function or thread is doing the locking. That alone makes a circular wait impossible.

Second, let the library handle the ordering for you with std::scoped_lock (C++17), which can lock several mutexes at once, safely, no matter what order you list them in at each call site:

void threadFunc1() {
    std::scoped_lock lock(mA, mB);   // locks both together -- deadlock-safe
    // ... use both resources ...
}

void threadFunc2() {
    std::scoped_lock lock(mB, mA);   // different order here, and it is still fine
    // ... use both resources ...
}

std::scoped_lock uses an internal algorithm that avoids ever holding one of the mutexes while blocked waiting for another, so this version cannot deadlock even though the two functions list the mutexes in opposite order.

9. Why engines don't spawn a thread per task: the thread pool

Given all these tools, a tempting idea is: for every piece of work — update this enemy, load that asset, compute this AI decision — just spin up a fresh std::thread. Real engines almost never do this, for two solid reasons.

First, creating an OS thread is not free. The operating system has to reserve a stack for it (often around a megabyte by default), set up scheduling data for it, and register it with the kernel. Doing that thousands of times a frame — one per enemy, say — can cost more time than the actual work.

Second, a CPU only has so many cores. If you create 500 threads on an 8-core machine, only 8 of them can truly run at the same instant; the rest are just sitting in line, and the OS has to keep switching which ones get a turn (a context switch), which itself costs time and can hurt cache performance (remember cache locality from the data structures chapter — jumping between many threads' working sets thrashes the cache the same way jumping between scattered memory does).

The fix engines use is a thread pool: create a small, fixed number of worker threads once, at startup — typically around the number of CPU cores — and reuse those same threads for the entire life of the program, feeding them work instead of creating new threads for every task.

10. Splitting work into tasks: the job system

A thread pool needs something to feed it. That something is a job (also called a task): a small, self-contained unit of work — usually a function plus the data it needs — small enough that many of them can be handed out and finished quickly. A job system is the machinery that holds a queue of jobs and lets the worker pool pull and run them.

The idea in practice: instead of "update all 10,000 enemies" being one giant piece of work run by one thread, you split it into, say, 10 jobs of 1,000 enemies each, and push all 10 into the queue. Whichever worker thread is free next grabs a job, runs it, and grabs another — so all your cores stay busy on the same big task together, without anyone creating a new thread. Many job systems also support dependencies — "do not start job C until jobs A and B are both finished" — so you can describe something like "physics must finish before the render job list is built" as a small graph of jobs, instead of manually joining specific threads.

Here is a small, simplified job queue to see the shape of it. It introduces one new tool, std::condition_variable, which lets a thread sleep efficiently until another thread wakes it up, instead of wasting CPU time checking "is there work yet?" in a loop (called busy-waiting).

#include <thread>
#include <mutex>
#include <queue>
#include <functional>
#include <condition_variable>
#include <vector>
#include <iostream>

std::mutex qm;
std::condition_variable cv;
std::queue<std::function<void()>> jobs;
bool stop = false;

void worker() {
    while (true) {
        std::function<void()> job;
        {
            std::unique_lock<std::mutex> lock(qm);
            cv.wait(lock, [] { return !jobs.empty() || stop; });  // sleep until told otherwise
            if (stop && jobs.empty()) return;
            job = jobs.front();
            jobs.pop();
        }
        job();   // run OUTSIDE the lock, so other workers can keep grabbing jobs meanwhile
    }
}

void pushJob(std::function<void()> j) {
    {
        std::lock_guard<std::mutex> lock(qm);
        jobs.push(j);
    }
    cv.notify_one();   // wake one sleeping worker
}

int main() {
    std::vector<std::thread> pool;
    for (int i = 0; i < 4; i++) pool.emplace_back(worker);   // 4 workers, created ONCE

    for (int i = 0; i < 8; i++)
        pushJob([i] { std::cout << "job " << i << " done\n"; });

    // (real code would wait for all 8 jobs to finish here before shutting down)

    {
        std::lock_guard<std::mutex> lock(qm);
        stop = true;
    }
    cv.notify_all();
    for (auto& t : pool) t.join();
}

Output: eight lines, "job 0 done" through "job 7 done" — but not necessarily in that order, since 4 workers are pulling from the same queue at once. One typical run:

job 0 done
job 2 done
job 1 done
job 3 done
job 5 done
job 4 done
job 6 done
job 7 done

Same idea as section 2's unordered output, just with more threads involved. The important part is not the order — it is that only 4 std::thread objects were ever created, no matter whether you push 8 jobs or 8,000. That is the whole point of separating "how many OS threads exist" (small and fixed) from "how many pieces of work there are" (as many as you like).

11. One thread that must never block: audio

One thread in a real engine plays by stricter rules than all the others: the audio thread (or audio callback). The operating system calls it on a tight, fixed schedule to hand over the next small chunk of sound samples — often just a few milliseconds' worth. Miss that deadline even once, and the speakers run out of data: the player hears an audible click, pop, or crackle. That is far more noticeable than a single dropped graphics frame, and it happens instantly, with no room to "catch up next frame."

Because of that deadline, the audio thread must never block — it must never do anything that could make it wait an unpredictable amount of time:

• It must not call lock() on a mutex that some other thread might be holding. If a lower-priority thread holds that mutex and the OS happens to pause that thread (for something unrelated), the audio thread can end up stuck waiting behind a thread that itself is not even running — a problem called priority inversion.
• It must not allocate memory with new or malloc. Memory allocators often use their own internal mutex, and even without one, allocation can take an unpredictable amount of time.
• It must not do file I/O, or wait on anything else that could stall.

Since the audio thread cannot use mutexes, real engines lean on exactly the lock-free tools from this chapter — std::atomic — plus patterns like double buffering: one thread writes fresh audio data into a "back" buffer while the audio thread safely reads a completed "front" buffer, and the two threads swap which buffer is which by atomically flipping a single pointer or index, never by locking anything.

The general lesson reaches beyond audio: any code with a hard real-time deadline should avoid mutexes on its critical path, and lean on atomics and careful data layout instead. Most of your gameplay code will never need this level of care — but knowing it exists, and why, is what lets you recognize the one or two threads in an engine where the normal "just use a mutex" advice from section 5 does not apply.

12. Glossary

• Thread — a single sequence of instructions the CPU runs, one after another; a program starts with one and can create more.
• Core — the physical CPU hardware that runs one thread's instructions at a time.
• std::thread — the C++ standard type that creates and runs a new thread.
• join() / detach() — wait for a thread to finish, or let it run fully independently; every std::thread must do one or the other.
• Data race — two or more threads accessing the same variable at once with no coordination, at least one a write; undefined behavior in C++.
• Critical section — a stretch of code that only one thread may run at a time.
• Mutex — a lock ("mutual exclusion") that only one thread can hold at once; lock()/unlock() mark a critical section.
• std::lock_guard — locks a mutex in its constructor and unlocks it automatically in its destructor (RAII), so a lock can never be forgotten.
• RAII — "Resource Acquisition Is Initialization": a constructor sets something up, its destructor cleans it up automatically no matter how the scope exits.
• std::atomic — a type whose operations happen as one indivisible step, using CPU instructions instead of a mutex; good for a single simple value.
• Deadlock — two or more threads each waiting on a lock the other holds; the program hangs forever with no crash.
• Circular wait — the pattern that causes deadlock: mutexes locked in opposite orders by different threads.
• std::scoped_lock — locks multiple mutexes together safely, regardless of the order they are listed in, avoiding deadlock.
• Thread pool — a small, fixed set of worker threads created once and reused for the life of the program, instead of creating a new thread per task.
• Job / task — a small, self-contained unit of work (function plus data) handed to a worker pool.
• Job system — the queue and scheduling logic that lets a fixed worker pool pull and run many jobs, sometimes with dependencies between them.
• std::condition_variable — lets a thread sleep efficiently until another thread wakes it, instead of busy-waiting.
• Busy-waiting — repeatedly checking "is there work yet?" in a loop, wasting CPU instead of sleeping.
• Priority inversion — a high-priority thread stuck waiting on a lock held by a lower-priority thread that is not currently running.
• Double buffering — one thread writes to an inactive buffer while another safely reads a complete one, swapped atomically, avoiding locks.

13. Exercises

#include <thread>
#include <iostream>

int health = 1000000;

void damage() {
    for (int i = 0; i < 500000; i++) {
        health--;
    }
}

int main() {
    std::thread t1(damage);
    std::thread t2(damage);
    t1.join();
    t2.join();
    std::cout << health << "\n";
}

#include <thread>
#include <atomic>
#include <iostream>

std::atomic<int> health{1000000};

void damage() {
    for (int i = 0; i < 500000; i++) {
        health--;   // atomic decrement
    }
}

int main() {
    std::thread t1(damage);
    std::thread t2(damage);
    t1.join();
    t2.join();
    std::cout << health << "\n";   // always 0
}

// std::mutex assetsMutex, sceneMutex;  declared elsewhere

void loadThread() {
    std::lock_guard<std::mutex> l1(assetsMutex);
    std::lock_guard<std::mutex> l2(sceneMutex);
    // ...
}

void unloadThread() {
    std::lock_guard<std::mutex> l1(sceneMutex);
    std::lock_guard<std::mutex> l2(assetsMutex);
    // ...
}

void loadThread()   { std::scoped_lock lock(assetsMutex, sceneMutex); /* ... */ }
void unloadThread() { std::scoped_lock lock(sceneMutex, assetsMutex); /* ... */ }

That is the core toolkit: a thread runs code independently, a data race is what happens when two threads touch the same memory without coordination, and std::mutex/std::lock_guard/std::atomic are the tools that add the missing coordination — each suited to a different situation. Deadlock is the danger that comes with locks themselves, avoided with a consistent lock order or std::scoped_lock. And the shape real engines actually use in production is not "one thread per task" but a small, fixed thread pool fed by a job system, with one thread — audio — that opts out of locking altogether because it cannot afford to wait. Carry the mental model forward: identify what is shared, protect it with the lightest tool that is still correct, and never guess about timing — race conditions and deadlocks earn a profiler and a stress test, not a guess.