3.2 Memory Management

Chapter 3.1 showed why the CPU cares about how data sits in memory — cache lines, locality, and why looping over a contiguous array beats chasing pointers around the heap. This chapter builds directly on that. You will see why malloc and new are not free, why the compiler quietly wastes memory through something called padding, why real engines reshape their data into a layout called Struct-of-Arrays, and how to write a tiny custom allocator yourself — the same trick engines like Unreal, Unity's native layer, and most AAA studios use so the game does not stall for memory in the middle of a frame.

Every section follows the same shape as before: small runnable C++ code, the real output, then a plain explanation of what happened. Type the examples in and run them yourself.

1. Recap: Stack vs Heap, and Why new and malloc Are Not Free

You already met the stack and the heap in the C chapters. Quick recap, because everything else in this chapter depends on it:

• The stack holds local variables. It is LIFO (last in, first out) — a function pushes its variables on entry and they pop off automatically on return. Allocating on the stack is just moving one pointer (the stack pointer) up or down. That is a handful of CPU instructions, so it is essentially free.
• The heap holds memory you ask for by hand with new or malloc, and you must give it back with delete or free. The heap is not one simple pointer — it is managed by a general-purpose allocator (part of the C++ runtime) that tracks which blocks are free and which are used, usually with a data structure like a free list (a list of free blocks).

#include <cstdio>

struct Particle { float x, y; };

void makeOnStack() {
    Particle p{1.0f, 2.0f};                       // lives on the stack
    printf("stack particle at %p\n", (void*)&p);
}                                                   // p is destroyed here, instantly, automatically

void makeOnHeap() {
    Particle* p = new Particle{1.0f, 2.0f};        // lives on the heap
    printf("heap particle at %p\n", (void*)p);
    delete p;                                       // must free by hand, or it leaks
}

int main() {
    makeOnStack();
    makeOnHeap();
}

Output (the exact addresses will differ on your machine — that is normal):

stack particle at 0x7ffde3a1c85c
heap particle at 0x55f2a1b0aeb0

Both particles get made and used exactly the same way from the caller's point of view. The difference is what happens underneath. makeOnStack just moves the stack pointer down by sizeof(Particle) bytes on entry and back up on return — no bookkeeping. makeOnHeap asks the allocator to search its free list for a block big enough, mark it used, and later, on delete, mark it free again and maybe merge it with neighbouring free blocks. In a program with many threads, that search is often protected by a lock, so two threads cannot corrupt the same free list at once — another cost the stack never pays.

Over the lifetime of a long-running program (a game can run for hours), repeatedly allocating and freeing different sizes leaves holes of unused memory scattered between used blocks. That is called fragmentation.

None of this makes new/malloc bad — you use them constantly, and for most code they are the right, simple choice. The problem is calling them every frame, in a hot loop (a "hot path" is code that runs extremely often, like once or many times per frame). That is where the search, the possible lock, and the slow drift toward fragmentation start to cost real milliseconds. Later sections show what engines do instead.

2. Alignment: Why the CPU Cares Where Data Starts

The CPU does not read memory one byte at a time when it can help it — it reads in chunks sized to match each type, and it wants those chunks to start at an address that is a multiple of the type's size. That required multiple is called the type's alignment. A 4-byte int wants to start at an address divisible by 4. An 8-byte double wants to start at an address divisible by 8. Reading a value that starts at a "wrong" address is called a misaligned access — on x86 it usually just costs extra cycles; on some other CPUs (and for some SIMD instructions, which you met in 3.1) it can outright crash.

C++ tells you a type's required alignment with alignof:

#include <cstdio>

int main() {
    printf("alignof(char)   = %zu\n", alignof(char));
    printf("alignof(int)    = %zu\n", alignof(int));
    printf("alignof(double) = %zu\n", alignof(double));
}

Output (typical on a 64-bit desktop or console):

alignof(char)   = 1
alignof(int)    = 4
alignof(double) = 8

A char can start anywhere (alignment 1). An int must start at a multiple of 4. A double must start at a multiple of 8. The compiler guarantees this automatically for every variable and every struct member — by inserting extra unused bytes where needed. That is padding, and it is the whole subject of the next section.

3. Padding: Why sizeof Is Bigger Than You Think

Because every member of a struct must start at an address matching its own alignment, the compiler sometimes has to leave gaps between members. Those gaps are pure waste — they hold no data, they are just there so the next member lands at a legal address. Look at this struct:

#include <cstdio>

struct Bad {
    char   a;   // 1 byte
    double b;   // 8 bytes
    char   c;   // 1 byte
};

int main() {
    printf("sizeof(Bad) = %zu\n", sizeof(Bad));
}

Output:

sizeof(Bad) = 24

Twenty-four bytes, even though a, b, and c only add up to 1 + 8 + 1 = 10 bytes of real data. Here is where the other 14 bytes go:

a takes offset 0. b is a double and needs to start at a multiple of 8, so the compiler skips offsets 1-7 (7 wasted bytes) and places b at offset 8. c follows immediately at offset 16. Then one more rule kicks in: the whole struct's size must be a multiple of the struct's own alignment (which is the largest alignment among its members — here, 8, from the double). That is needed so that if you make an array of Bad, every element in the array stays correctly aligned too. 17 is not a multiple of 8, so the compiler pads out to 24.

4. Reordering Fields to Shrink a Struct

Take the same three fields and just change the order — biggest field first:

#include <cstdio>

struct Good {
    double b;   // 8 bytes
    char   a;   // 1 byte
    char   c;   // 1 byte
};

static_assert(sizeof(Good) == 16, "Good grew - check the field order again");

int main() {
    printf("sizeof(Good) = %zu\n", sizeof(Good));
}

Output:

sizeof(Good) = 16

b sits at offset 0-7 (already aligned, no padding needed before it). a follows at offset 8, c at offset 9. Total real data so far: 10 bytes, ending at offset 10. Round up to the next multiple of 8 (the struct's alignment) and you land on 16. Same three fields, same data, but Good is 16 bytes instead of Bad's 24 — a third smaller, just from reordering. Multiply that saving by a million game objects sitting in memory and it adds up to real bytes, and — just as important — fewer cache lines to load when you stream through an array of them (the cache-line idea from 3.1).

The rule of thumb: sort fields from largest alignment to smallest (roughly: biggest type to smallest type), and group same-size fields together. The static_assert line above is a good habit too — it fails to compile if someone later adds a field and accidentally grows the struct, so you find out immediately instead of six months later in a profiler.

5. Array-of-Structs vs Struct-of-Arrays: a Particle System

Padding is about the layout of one object. This section is about the layout of many objects — thousands of particles for an explosion or a spell effect, all needing the same fields (position, velocity, remaining lifetime) updated every frame. There are two natural ways to store them.

Array-of-Structs (AoS): one struct per particle

This is the layout every beginner reaches for first — it matches how you would describe "a particle" as one object:

#include <vector>

struct Particle {
    float x, y, z;      // position
    float vx, vy, vz;   // velocity
    float life;         // seconds remaining
};

void updatePositions(std::vector<Particle>& particles, float dt) {
    for (Particle& p : particles) {
        p.x += p.vx * dt;
        p.y += p.vy * dt;
        p.z += p.vz * dt;
    }
}

This has no printed output by itself — it is a worked trace instead. Particle is seven floats, each 4 bytes, all with the same 4-byte alignment, so there is no padding: sizeof(Particle) = 28 bytes exactly. std::vector<Particle> stores all 28-byte particles back to back — one array, but every element is the whole bundle of seven fields glued together.

Struct-of-Arrays (SoA): one array per field

Flip it inside out: instead of one array of particle-structs, keep seven separate arrays, one per field, all the same length:

#include <vector>

struct ParticleSystem {
    std::vector<float> x, y, z;      // position, one array per axis
    std::vector<float> vx, vy, vz;   // velocity, one array per axis
    std::vector<float> life;         // seconds remaining
};

void updatePositions(ParticleSystem& ps, float dt) {
    size_t count = ps.x.size();
    for (size_t i = 0; i < count; ++i) {
        ps.x[i] += ps.vx[i] * dt;
        ps.y[i] += ps.vy[i] * dt;
        ps.z[i] += ps.vz[i] * dt;
    }
}

Particle 5's data is not glued together anywhere in memory anymore — its x lives in the x array at index 5, its vx lives in the vx array at index 5, and so on. The two versions store exactly the same information and produce exactly the same result. The difference is entirely about memory layout:

6. Why SoA Wins: Cache Lines and SIMD

Recall from 3.1: the CPU never fetches a single value from RAM on its own — it always pulls in a whole cache line at once (commonly 64 bytes on desktop and console CPUs). How much useful work you get from each cache line depends entirely on how the data is arranged.

#include <cstdio>

struct Particle { float x, y, z, vx, vy, vz, life; };

int main() {
    printf("sizeof(Particle) = %zu\n", sizeof(Particle));
}

Output:

sizeof(Particle) = 28

Now trace updatePositions from the previous section for both layouts, assuming a 64-byte cache line and 1,000 particles:

• AoS: updating x for one particle pulls in that particle's whole 28-byte footprint, whether the loop wants vy, vz, or life this pass or not — those bytes ride along regardless. For 1,000 particles, the loop touches roughly 1,000 * 28 = 28,000 bytes, about 438 cache lines.
• SoA: the x array is 1,000 contiguous floats and nothing else. The same loop touches roughly 1,000 * 4 = 4,000 bytes, about 63 cache lines — roughly 7 times fewer than AoS. That factor of 7 is not a coincidence: Particle has 7 float fields of equal size, and this loop only needed 1 of them.

The gap gets bigger with real-world particle structs, which often carry a color, a texture index, an animation frame, and more — fields this particular loop never touches but AoS forces you to drag along anyway.

This also ties straight back to SIMD (single instruction, multiple data — one CPU instruction that operates on several values at once, from 3.1). A SIMD instruction wants to load, say, 8 contiguous floats into one register and add them all in a single step. The SoA x array is exactly that shape — 8 contiguous floats, load them directly. The AoS layout interleaves x with six other fields, so getting 8 x values into one register needs a special (and slower, sometimes unavailable) "gather" instruction instead of a plain contiguous load. SoA is not just cache-friendly — it is the layout SIMD code wants to see.

7. Why Engines Don't Call new/delete in the Hot Path

A game running at 60 frames per second has about 16.6 milliseconds to produce each frame — physics, AI, animation, rendering, everything. Section 1 already listed what a general-purpose new/malloc does under the hood: search a free list, maybe take a lock, occasionally ask the operating system for more memory (a system call, which is far slower than ordinary instructions). None of that is disastrous for a single call. It becomes a problem when it happens hundreds or thousands of times a frame.

#include <vector>

struct Bullet { float x, y, vx, vy; };

void spawnBullet(std::vector<Bullet*>& bullets) {
    Bullet* b = new Bullet{};    // a heap allocation on the hot path
    bullets.push_back(b);
}

This is a worked trace, not a printed output: imagine a machine gun that fires 20 bullets a second, and each bullet also gets deleted a moment later when it expires. That is 20 heap allocations and 20 frees every second from one weapon alone, repeated for every enemy and every player, for the entire length of a play session that might run for hours. Individually each call might take a fraction of a microsecond — but it adds up, it can randomly spike (a worst-case search through a messy free list is much slower than the average case), and it slowly fragments the heap the longer the game runs. A single slow frame is a visible stutter the player notices; a game engine cares about the worst case, not just the average.

The fix is not "avoid the heap entirely" — it is "stop using a general-purpose allocator for a job whose allocation pattern you already know in advance." If you know a batch of allocations all die together, or that you are always allocating the exact same size over and over, you can write an allocator built for exactly that pattern, and it can be far simpler — and therefore far faster and more predictable — than a general-purpose one. The rest of this chapter builds two of the most common ones engines use.

8. Building a Bump (Arena) Allocator

The simplest custom allocator is the arena allocator, also called a bump allocator (or "linear allocator"). The idea: grab one big block of memory up front. To "allocate", just hand out the next unused slice and move a pointer forward past it — that is the "bump". To free, do not free anything individually — instead, throw away the whole block's contents at once by moving the pointer back to the start. No search, no free list, no bookkeeping per allocation.

#include <cstdio>
#include <cstdint>
#include <cstdlib>

class ArenaAllocator {
public:
    explicit ArenaAllocator(size_t sizeBytes) {
        m_begin   = static_cast<uint8_t*>(std::malloc(sizeBytes));
        m_current = m_begin;
        m_end     = m_begin + sizeBytes;
    }

    ~ArenaAllocator() {
        std::free(m_begin);
    }

    void* allocate(size_t bytes) {
        if (m_current + bytes > m_end) {
            return nullptr;              // arena is full
        }
        void* result = m_current;
        m_current += bytes;              // the "bump"
        return result;
    }

    void reset() {
        m_current = m_begin;             // "free" everything in one step
    }

private:
    uint8_t* m_begin;
    uint8_t* m_current;
    uint8_t* m_end;
};

int main() {
    ArenaAllocator arena(64);            // a tiny 64-byte arena

    void* a = arena.allocate(16);
    void* b = arena.allocate(16);
    void* c = arena.allocate(16);

    printf("a = %p\n", a);
    printf("b = %p\n", b);
    printf("c = %p\n", c);
    printf("b - a = %td bytes\n", (uint8_t*)b - (uint8_t*)a);

    arena.reset();                       // everything "freed" at once, no destructors run

    void* d = arena.allocate(16);
    printf("d = %p (same address as a: %s)\n", d, d == a ? "yes" : "no");
}

Output (your exact hex addresses will differ, but the pattern will not):

a = 0x55f2a1b0aeb0
b = 0x55f2a1b0aec0
c = 0x55f2a1b0aed0
b - a = 16 bytes
d = 0x55f2a1b0aeb0 (same address as a: yes)

b - a = 16 bytes proves each allocate call just moved the pointer forward by exactly the requested size — no searching, no metadata written per allocation, just addition. And after reset(), d lands on the exact same address as a did, because reset() did not erase anything — it just moved m_current back to m_begin, so the next allocate hands out that same memory again.

Allocating real objects, not just bytes

allocate() only hands you raw, uninitialized bytes — it does not know how to build a Particle or call a constructor. To build a real object inside that memory, use placement new: a special form of new that constructs an object at an address you already have, instead of asking the heap for a new one.

#include <cstdio>
#include <new>

struct Particle {
    float x, y, z;
};

int main() {
    ArenaAllocator arena(1024);

    void* mem = arena.allocate(sizeof(Particle));
    Particle* p = new (mem) Particle{1.0f, 2.0f, 3.0f};   // construct IN this memory

    printf("particle: %.1f, %.1f, %.1f\n", p->x, p->y, p->z);

    arena.reset();   // Particle's destructor is NOT called here
}

Output:

particle: 1.0, 2.0, 3.0

new (mem) Particle{...} does not allocate anything — it runs Particle's constructor directly on the bytes at mem, which the arena already owns. This is fine for a Particle, which owns no other resources (no heap pointers of its own, nothing that needs cleanup). It would be dangerous for a type that owns a resource (say, a file handle or another heap allocation), because reset() does not call destructors — it just wipes the pointer back to the start. That is a deliberate trade: arenas are for data with no cleanup needs and a shared lifetime, which describes an enormous amount of game data (particles, per-frame draw commands, temporary AI query results).

9. The Per-Frame Scratch Allocator Pattern

The arena allocator's "free everything at once" behavior is oddly perfect for one very common situation in a game: work that is only needed for the current frame and can be thrown away the moment the next frame starts. Engines call this a per-frame scratch allocator (or "frame allocator", or "linear allocator" used this specific way).

The pattern: create one arena when the game starts, sized generously (a few megabytes is common). At the very start of every frame, call reset(). During the frame, any system that needs short-lived scratch memory — building a list of visible objects for this frame, formatting a debug string, holding temporary pathfinding search data — allocates from that same arena instead of calling new. Nobody calls an individual free. When the frame ends and the next one begins, reset() throws it all away in one step, ready to be reused.

Compare the cost to calling new/delete for the same scratch data every frame: with the arena, every allocation this frame is just a pointer bump, and freeing the entire frame's worth of scratch data costs exactly one instruction (m_current = m_begin), no matter how many allocations happened. There is also no fragmentation risk, because the lifetime of everything in the arena is identical — it all dies at the same instant, every frame, forever. That is the trade a general-purpose allocator cannot make, because it has no idea in advance which allocations will be freed together.

10. Pool Allocators: Fixed-Size Slots

An arena has one weakness: you cannot free just one thing out of the middle — everything shares one lifetime. Games are full of objects that do not share a lifetime: enemies spawn and die at unrelated times, bullets fire and expire independently, network packets arrive and get consumed one at a time. For that pattern, engines reach for a pool allocator.

A pool pre-allocates room for a fixed number of same-size slots up front — enough for, say, 500 enemies. Free slots are threaded into a free list, a linked list built cleverly inside the unused slots themselves: each free slot's first few bytes store a pointer to the next free slot, so no extra memory is needed just to track which slots are free.

#include <cstdio>
#include <cstdint>
#include <cstdlib>

class PoolAllocator {
public:
    PoolAllocator(size_t slotSize, size_t slotCount) : m_slotSize(slotSize) {
        m_memory   = static_cast<uint8_t*>(std::malloc(slotSize * slotCount));
        m_freeList = nullptr;
        for (size_t i = 0; i < slotCount; ++i) {
            void* slot = m_memory + i * slotSize;
            *reinterpret_cast<void**>(slot) = m_freeList;   // store "next free" IN the slot
            m_freeList = slot;
        }
    }

    void* allocate() {
        if (!m_freeList) {
            return nullptr;                     // pool is full
        }
        void* slot = m_freeList;
        m_freeList = *reinterpret_cast<void**>(slot);
        return slot;
    }

    void deallocate(void* slot) {
        *reinterpret_cast<void**>(slot) = m_freeList;
        m_freeList = slot;
    }

private:
    uint8_t* m_memory;
    void*    m_freeList;
    size_t   m_slotSize;
};

struct Enemy { float x, y, z; int health; };   // 16 bytes - room enough for a pointer

int main() {
    PoolAllocator pool(sizeof(Enemy), 4);       // 4 slots, each big enough for one Enemy

    void* s0 = pool.allocate();
    void* s1 = pool.allocate();
    pool.deallocate(s0);
    void* s2 = pool.allocate();                 // reuses s0's slot

    printf("s0 = %p\n", s0);
    printf("s2 = %p (same address as s0: %s)\n", s2, s2 == s0 ? "yes" : "no");
}

Output:

s0 = 0x55f2a1b0af00
s2 = 0x55f2a1b0af00 (same address as s0: yes)

Both allocate() and deallocate() are O(1) — pop or push the front of a linked list, no searching. Unlike a general-purpose allocator, a pool never fragments, because every slot is exactly the same size: any free slot can satisfy any request, so there is never a "hole too small" problem.

11. Choosing the Right Allocator

None of these replace each other — a real engine uses all of them side by side, picked per situation:

• Stack (automatic variables) — the default for anything that does not need to outlive the current function. Fastest possible option, and you do not write any allocator code at all.
• Arena / bump allocator — many allocations that all die together: per-frame scratch data, temporary buffers while loading a level, the working memory for a single pathfinding search. O(1) to allocate, O(1) to free everything at once.
• Pool allocator — many objects of the same size with independent lifetimes: bullets, enemies, particles kept as individual objects, network packets. O(1) to allocate and free one at a time, with zero fragmentation.
• General-purpose new/malloc — rare, irregular-sized, long-lived allocations, and anything outside the hot path: building a level's data once at load time, constructing a UI, a one-off manager object. Perfectly fine there.

The recurring idea across this whole chapter: know your data's shape (padding, AoS vs SoA) and know your data's lifetime (arena vs pool vs general-purpose), and pick tools that match both. That is what "allocators the way engines do it" means — not one universal allocator, but a small toolbox of specialists, each simple because it only has to handle one specific pattern well.

12. Glossary

• Allocator — code responsible for handing out and reclaiming chunks of memory.
• Alignment — the address multiple a type must start at (e.g. a double needs a multiple of 8).
• Padding — unused filler bytes the compiler inserts so members and whole structs land on legal, aligned addresses.
• AoS (Array-of-Structs) — one array holding whole objects back to back, all fields of an object glued together.
• SoA (Struct-of-Arrays) — one separate array per field, so all values of the same field sit together in memory.
• Cache line — the fixed-size chunk (commonly 64 bytes) the CPU fetches from RAM at once, covered in 3.1.
• SIMD — single instruction, multiple data; one instruction that processes several contiguous values at once.
• Fragmentation — free memory scattered into holes too small individually to satisfy a new request, even though the total free space is enough.
• Hot path — code that runs very often (every frame, or many times per frame), where small costs add up fast.
• Arena / bump allocator — hands out memory by moving a pointer forward; frees everything at once by resetting that pointer.
• Per-frame scratch allocator — an arena reset at the start of every frame, used for that frame's short-lived temporary data.
• Pool allocator — pre-allocated fixed-size slots handed out and returned one at a time via a free list; no fragmentation, O(1) allocate/free.
• Free list — a linked list of currently-unused blocks or slots, often stored inside the unused memory itself.
• Placement new — a form of new that constructs an object at a memory address you already own, instead of allocating new memory.

13. Exercises

#include <cstdio>

struct Enemy {
    bool   isAlive;
    double health;
    char   team;
    int    id;
};

int main() {
    printf("sizeof(Enemy) = %zu\n", sizeof(Enemy));
}

#include <cstdio>

struct Enemy {
    double health;
    int    id;
    bool   isAlive;
    char   team;
};

int main() {
    printf("sizeof(Enemy) = %zu\n", sizeof(Enemy));
}

sizeof(Enemy) = 16

#include <vector>

struct Enemy {
    float x, y;
    float health;
    int   id;
};

// TODO: struct EnemySystem with parallel arrays instead of a vector<Enemy>
// TODO: void damageAll(EnemySystem& es, float amount);

#include <vector>
#include <cstdio>

struct EnemySystem {
    std::vector<float> x, y;
    std::vector<float> health;
    std::vector<int>   id;
};

void damageAll(EnemySystem& es, float amount) {
    for (size_t i = 0; i < es.health.size(); ++i) {
        es.health[i] -= amount;
    }
}

int main() {
    EnemySystem es;
    es.x      = {0.0f, 1.0f};
    es.y      = {0.0f, 1.0f};
    es.health = {100.0f, 50.0f};
    es.id     = {1, 2};

    damageAll(es, 10.0f);

    printf("health[0] = %.1f\n", es.health[0]);
    printf("health[1] = %.1f\n", es.health[1]);
}

health[0] = 90.0
health[1] = 40.0

// Starting point: same ArenaAllocator as section 8 (m_begin, m_current, m_end).
// TODO: add this method.
//   void* allocate(size_t bytes, size_t alignment);
// It must round m_current UP to the next multiple of "alignment"
// before returning it, and must still refuse to overrun m_end.

#include <cstdio>
#include <cstdint>
#include <cstdlib>

class ArenaAllocator {
public:
    explicit ArenaAllocator(size_t sizeBytes) {
        m_begin   = static_cast<uint8_t*>(std::malloc(sizeBytes));
        m_current = m_begin;
        m_end     = m_begin + sizeBytes;
    }

    ~ArenaAllocator() {
        std::free(m_begin);
    }

    void reset() {
        m_current = m_begin;
    }

    // alignment MUST be a power of two (1, 2, 4, 8, 16, ...) - every real
    // C++ alignment requirement is a power of two, so this is always safe.
    void* allocate(size_t bytes, size_t alignment) {
        uintptr_t rawAddr     = reinterpret_cast<uintptr_t>(m_current);
        uintptr_t alignedAddr = (rawAddr + (alignment - 1)) & ~(alignment - 1);
        size_t padding        = static_cast<size_t>(alignedAddr - rawAddr);

        if (m_current + padding + bytes > m_end) {
            return nullptr;             // not enough room, even counting padding
        }

        m_current += padding;           // skip the padding bytes
        void* result = m_current;       // this address is now aligned
        m_current += bytes;
        return result;
    }

private:
    uint8_t* m_begin;
    uint8_t* m_current;
    uint8_t* m_end;
};

int main() {
    ArenaAllocator arena(64);

    void* p1 = arena.allocate(1, 1);    // 1 byte, no alignment need
    void* p2 = arena.allocate(8, 8);    // 8 bytes, must land on a multiple of 8

    printf("p1 = %p\n", p1);
    printf("p2 = %p\n", p2);
    printf("p2 is 8-byte aligned: %s\n",
           (reinterpret_cast<uintptr_t>(p2) % 8 == 0) ? "yes" : "no");
}

p1 = 0x55f2a1b0b000
p2 = 0x55f2a1b0b008
p2 is 8-byte aligned: yes