15.6 SIMD & Cache (deep roles)

Phase 15 · Optimization & Mobile · Study time: 30–50 h

Squeezing the CPU — cache-friendly data layout (Struct-of-Arrays) and SIMD vectorization, for engine and performance specialists.

In the C memory chapter you learned that memory is just a giant array of numbered bytes, and a pointer is a number that names one of those bytes. Arrays store their elements one after another in that byte array, with no gaps. That was about correctness — knowing what a piece of memory actually is. This chapter is about speed — what happens when your code actually asks the CPU to read and write those bytes, millions of times a frame.

1. Why This Chapter: From Bytes to Speed

Here is the surprising part. Two pieces of code can do the exact same math — the same number of additions and multiplications — and still run 5 to 30 times apart in speed, just because one of them asks for memory in an order the CPU likes and the other does not. This is not a minor detail. In real game engines, memory access patterns are often a bigger performance lever than clever algorithms. A physics engine, a particle system, or a skinning system that "does the same work" but touches memory badly can lose an entire frame's time budget to nothing but waiting.

Two techniques dominate this chapter: making memory access cache-friendly (so the CPU rarely has to wait for RAM), and using SIMD (Single Instruction, Multiple Data — getting one CPU instruction to do 4 or 8 pieces of math at once instead of one). Both are used constantly in particle systems, physics solvers, skinning, and culling. Both are also the reason the Entity-Component-System (ECS) pattern exists, which you will meet properly in Phase 16 — this chapter explains the hardware reason ECS is shaped the way it is.

2. The Memory Hierarchy: Registers, L1, L2, L3, RAM

A CPU core cannot read a value straight out of RAM (main memory) for every single instruction — RAM is much too slow compared to how fast a modern core can execute instructions. So between the core and RAM sit several layers of smaller, faster memory called caches. Each layer is a copy of a small part of RAM, kept close to the core.

+-----------+ | REGISTERS | smallest, fastest (part of the CPU core) +-----------+ +-------------+ | L1 CACHE | small, very fast (per core) +-------------+ +-----------------+ | L2 CACHE | bigger, fast (per core) +-----------------+ +-----------------------+ | L3 CACHE | big, slower (shared by all cores) +-----------------------+ +-----------------------------------+ | MAIN MEMORY (RAM) | huge, slowest (shared) +-----------------------------------+

The layers closest to the core are tiny and extremely fast; the layers farther away are bigger and slower. L1, L2, and L3 stand for "Level 1/2/3 cache" — L1 is closest and smallest, L3 is farthest (but still much faster than RAM) and largest, usually shared by every core on the chip.

Rough access costs

The exact numbers vary by CPU model and generation, but the ratios between levels stay roughly similar across most modern desktop, console, and phone CPUs. As a rule of thumb, in CPU clock cycles:

level typical size rough latency ----- ------------ ------------- register a few bytes ~0-1 cycle L1 cache 32-64 KB per core ~4 cycles L2 cache 256 KB-1 MB per core ~12 cycles L3 cache several MB, shared ~40 cycles RAM many GB, shared ~200 cycles

A single add or multiply instruction takes roughly 1 cycle of throughput. That means a cache miss (the value was not already in cache) that goes all the way to RAM costs about as much time as 100 to 200+ ordinary arithmetic instructions. If your inner loop misses the cache on every iteration, the CPU spends almost all of its time sitting idle, waiting for bytes to arrive — not computing anything.

Tip These cycle counts do not need to be memorized exactly. What matters is the shape: each level is roughly 3-5x slower than the one above it, and the jump from L3 to RAM is the big one — about 5x worse than L3 alone.

3. Cache Lines: Memory Arrives 64 Bytes at a Time

Caches do not fetch memory one byte, or one float, at a time. Every fetch pulls in a fixed-size chunk called a cache line. On essentially every modern desktop, console, and mobile CPU, a cache line is 64 bytes.

So if you read a single 4-byte float at some address, the hardware actually pulls in the entire 64-byte-aligned chunk that contains it — 16 floats (or 8 doubles), whether you asked for the other 15 or not.

one 64-byte cache line, holding 16 floats: +----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+ | f0 | f1 | f2 | f3 | f4 | f5 | f6 | f7 | f8 | f9 |f10 |f11 |f12 |f13 |f14 |f15 | +----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+ \_________________________ one 64-byte cache line ___________________________/ reading f0 alone pulls in f0..f15 for free -- they all arrive together

This has a direct consequence for how you should write loops. If the next value you read is the next float in memory, it is very likely already sitting in the cache line you just paid for — reading it costs an L1 hit, about 4 cycles. If the next value you read is somewhere far away, that is a brand new cache line, possibly a full trip to RAM, about 200 cycles.

Sequential access is fast because it reuses cache lines you already fetched. Random or scattered access is slow because nearly every access needs a fresh cache line, and once your working set no longer fits in the cache, most of those fresh fetches go all the way to RAM.

Tip You do not need to detect the cache line size at runtime for this chapter. 64 bytes is a safe assumption for practically every CPU you will target — x86 (Intel/AMD), Apple Silicon, and mobile ARM chips.

4. Sequential Access Wins: Row-Major vs Column-Major

A 2D array is really a 1D array in disguise. In C and C++, a row-major matrix stores matrix[row][col] at the flat index row * width + col — so row 0 comes first in memory, in order, then row 1, then row 2, and so on.

row-major traversal (matches memory order): matrix in memory: [ (0,0) (0,1) (0,2) (0,3) | (1,0) (1,1) (1,2) (1,3) | (2,0) ... row 0 row 1 row 2 for row in 0..N: for col in 0..N: visit matrix[row][col] -- walks straight along memory column-major traversal (fights memory order): for col in 0..N: for row in 0..N: visit matrix[row][col] -- jumps N floats (one whole row) every step
#include <cstdio>
#include <chrono>
#include <vector>

const int N = 4096;

int main() {
    std::vector<float> matrix(N * N, 1.0f);

    // row-major: inner loop walks along memory (fast)
    auto t0 = std::chrono::high_resolution_clock::now();
    double sumRow = 0.0;
    for (int row = 0; row < N; row++) {
        for (int col = 0; col < N; col++) {
            sumRow += matrix[row * N + col];
        }
    }
    auto t1 = std::chrono::high_resolution_clock::now();

    // column-major: inner loop jumps N floats every step (slow)
    double sumCol = 0.0;
    for (int col = 0; col < N; col++) {
        for (int row = 0; row < N; row++) {
            sumCol += matrix[row * N + col];
        }
    }
    auto t2 = std::chrono::high_resolution_clock::now();

    double msRow = std::chrono::duration<double, std::milli>(t1 - t0).count();
    double msCol = std::chrono::duration<double, std::milli>(t2 - t1).count();

    printf("row-major:    %.2f ms  (sum = %.0f)\n", msRow, sumRow);
    printf("column-major: %.2f ms  (sum = %.0f)\n", msCol, sumCol);
}

Reading the timing code

Both loops do the exact same thing mathematically: sum every element of the same matrix, once each. The only difference is the order the two nested loops visit the indices in. The row-major version's inner loop (col) walks straight along memory, one float after another — it fully uses every cache line it fetches. The column-major version's inner loop (row) jumps N floats forward every step — for N = 4096, that is 16 KB between one access and the next, far bigger than a cache line and usually bigger than L1 or L2 entirely. Almost every access in the column-major loop is a fresh cache line fetch.

Measured on a typical desktop CPU, with N = 4096 (a 4096 x 4096 matrix of floats, 64 MB total — far bigger than any cache):

row-major:    14.20 ms  (sum = 16777216)
column-major: 168.53 ms  (sum = 16777216)

Same math, same number of additions, same final sum — almost 12x slower just from loop order. This is entirely a memory story, not a math story.

Common mistake Do not assume the compiler will fix a bad loop order for you. Compilers can reorder tiny, provably-safe loops, but for large nested loops over arrays — especially ones passed through pointers or references where aliasing is possible — the compiler usually cannot prove it is safe to change the order, so it does not. Getting the loop order right is on you.

5. Array of Structs vs Struct of Arrays

Most beginners write data the same way: a struct or class holding every field an object needs, stored in an array. This is called Array of Structs, or AoS.

AoS -- Array of Structs (one struct per particle): struct Particle { float x, y, z; float life; }; memory: [x0 y0 z0 life0][x1 y1 z1 life1][x2 y2 z2 life2]... \______particle 0______/\______particle 1______/ reading "life" for every particle also drags x, y, z along, whether you need them right now or not SoA -- Struct of Arrays (one array per field, across ALL particles): struct Particles { float x[N], y[N], z[N], life[N]; }; memory: [x0 x1 x2 x3 ...][y0 y1 y2 y3 ...][z0 z1 z2 ...][life0 life1 life2 ...] reading "life" for every particle touches ONLY the life array -- every byte pulled into cache is a byte you actually use

AoS matches how we think about objects — "a particle has a position, a velocity, a color" — and it is fine for code that touches most or all of an object's fields at once. The problem shows up when you only need one field across many objects. Because every field of a particle sits right next to its other fields in memory, reading just life for particle 0 still pulls x, y, z, vx, vy, vz, the whole color, rotation, and scale into the cache along with it — bytes you paid for but never use.

Struct of Arrays, or SoA, flips the layout: one array per field, across all objects. Here is a concrete version with a size closer to a real particle: 14 floats per particle (56 bytes), where the only thing this particular function needs is life.

struct ParticleAoS {
    float x, y, z;       // position
    float vx, vy, vz;    // velocity
    float r, g, b, a;    // color
    float rotation;
    float scale;
    float life;           // remaining lifetime -- the ONLY field this example needs
    float maxLife;
};

float sumLifeAoS(const std::vector<ParticleAoS>& particles) {
    float total = 0.0f;
    for (const auto& p : particles) {
        total += p.life;
    }
    return total;
}
struct ParticleSystemSoA {
    std::vector<float> x, y, z;
    std::vector<float> vx, vy, vz;
    std::vector<float> r, g, b, a;
    std::vector<float> rotation;
    std::vector<float> scale;
    std::vector<float> life;      // the only array this example touches
    std::vector<float> maxLife;
};

float sumLifeSoA(const ParticleSystemSoA& particles) {
    float total = 0.0f;
    for (float l : particles.life) {
        total += l;
    }
    return total;
}

Measured on 2,000,000 particles, all with life = 1.0:

int main() {
    const int N = 2000000;
    std::vector<ParticleAoS> aos(N);
    for (auto& p : aos) p.life = 1.0f;

    ParticleSystemSoA soa;
    soa.life.assign(N, 1.0f);

    auto t0 = std::chrono::high_resolution_clock::now();
    float totalAoS = sumLifeAoS(aos);
    auto t1 = std::chrono::high_resolution_clock::now();
    float totalSoA = sumLifeSoA(soa);
    auto t2 = std::chrono::high_resolution_clock::now();

    printf("AoS sumLife: %.2f ms (total = %.0f)\n",
           std::chrono::duration<double, std::milli>(t1 - t0).count(), totalAoS);
    printf("SoA sumLife: %.2f ms (total = %.0f)\n",
           std::chrono::duration<double, std::milli>(t2 - t1).count(), totalSoA);
}
AoS sumLife: 5.10 ms (total = 2000000)
SoA sumLife: 0.62 ms (total = 2000000)

The AoS version reads 56 bytes per particle to get 4 useful bytes — 14x more data than necessary streaming through the memory system. The SoA version reads exactly the 4 bytes it needs, packed tightly with 15 other particles' life values in the very same cache line. About 8x faster here, for reading a single field out of a moderately sized struct — the gap gets even bigger with bigger structs or more unused fields.

6. SoA and Why ECS Exists

The AoS vs SoA choice does not stop at particles. Zoom out to an entire game world and you get the same problem at a bigger scale: a traditional GameObject holds position, velocity, health, a mesh reference, AI state, and more, all in one heap-allocated object. A system that only cares about position and velocity — like a movement system — still has to jump to wherever each GameObject happens to live on the heap.

traditional (one big object per entity, scattered on the heap): GameObject A (heap addr 0x1000): position, velocity, health, mesh, ai state... GameObject B (heap addr 0x9F40): position, velocity, health, mesh, ai state... GameObject C (heap addr 0x2AA0): position, velocity, health, mesh, ai state... a "move everything" system jumps all over the heap to find position ECS (one tightly packed array per component, across ALL entities): Position array: [posA][posB][posC][posD][posE]... Velocity array: [velA][velB][velC][velD][velE]... Health array: [hpA ][hpB ][hpC ][hpD ][hpE ]... a "move everything" system streams straight through Position and Velocity only -- exactly the SoA pattern from section 5, applied to a whole game world

The Entity-Component-System (ECS) pattern, which you will meet properly in Phase 16, is SoA applied to an entire game world. Instead of one object per entity holding every field, ECS stores one tightly packed array per component type, across every entity that has that component. A movement system that only needs Position and Velocity streams straight through those two arrays and never touches health, AI state, or anything else — the exact same cache-line benefit as the particle example, just applied at the scale of thousands of entities instead of one particle system.

This is the hardware reason ECS exists. It is not merely a "cleaner" way to organize code — SoA-style component arrays are dramatically more cache-friendly than scattered objects, for exactly the reasons in this chapter.

Tip SoA is not an all-or-nothing rule for your whole codebase. Most engines use it only for hot, large-N, per-frame data — particles, transforms, bone matrices, physics bodies. Rarely-touched gameplay objects (a quest giver, a menu screen) gain nothing from SoA and are usually left as ordinary objects.

7. Prefetching and Predictable Access Patterns

Reusing cache lines is not the only reason sequential access is fast. Modern CPUs also contain a hardware prefetcher — circuitry that watches the pattern of addresses your code accesses. If it notices a steady stride (say, +64 bytes every access), it starts pulling the next few cache lines into cache before your code even asks for them, hiding RAM latency behind work the core is already doing.

memory: [line0][line1][line2][line3][line4][line5]... ^ core is reading here now hardware prefetcher notices the steady stride and starts pulling line3, line4, line5 into cache BEFORE the core asks for them, hiding the RAM latency behind work the core is already doing

This is another reason a simple loop over an array is fast: not only does it reuse cache lines, the prefetcher can predict exactly what address comes next and get a head start. Random access — following a linked list, or indexing through a shuffled index array — defeats the prefetcher completely. It has no steady stride to learn from, so every single access pays the full miss cost with no head start.

Compilers and hardware prefetchers handle simple loops well on their own. Manual prefetch hints exist (__builtin_prefetch in GCC/Clang, intrinsics like _mm_prefetch elsewhere) for cases the hardware cannot predict by itself — for example, walking a tree or linked list where you know which node you will need a few steps ahead:

// usually unnecessary -- shown only to see what it looks like
for (int i = 0; i < n; i++) {
    __builtin_prefetch(&data[i + 16], 0, 1); // hint: we will read this soon
    total += heavyWorkOn(data[i]);
}

For plain loops over arrays, like everything else in this chapter, you almost never need this — the hardware prefetcher already does it for you. Reach for manual prefetching only after profiling shows a specific loop is stalling on memory and the access pattern is not a simple array walk.

8. False Sharing: When Threads Fight Over a Cache Line

Cache-line reuse is great for a single core, but it creates a new problem once multiple cores are involved. Cache coherency (keeping every core's cached copy of data consistent) is tracked at the granularity of a whole cache line, not individual bytes. If two threads on two different cores write to two different variables that happen to land in the same 64-byte cache line, the hardware treats it as contention — even though the two variables never actually overlap. This is called false sharing.

Core A Core B +-----------+ +-----------+ | counterA | (write) | counterB | (write) +-----------+ +-----------+ \ / \__ same 64-byte cache line __/ Core A writes counterA: Core B's cached copy of the line is invalidated Core B writes counterB: Core A's cached copy of the line is invalidated result: the cache line keeps bouncing between the two cores (slow)

Every time core A writes counterA, it invalidates core B's cached copy of that entire line, forcing core B to refetch it. Every time core B writes counterB, the same thing happens in reverse. The line bounces back and forth between the two cores' caches — each bounce costs roughly as much as a cache miss to a shared cache or RAM, even though neither thread is touching the other's variable.

#include <atomic>
#include <thread>
#include <chrono>
#include <cstdio>

struct CountersBad {
    std::atomic<long> counterA{0};
    std::atomic<long> counterB{0};
    // counterA and counterB usually land in the SAME cache line
};

struct CountersGood {
    alignas(64) std::atomic<long> counterA{0};
    alignas(64) std::atomic<long> counterB{0};
    // 64-byte alignment forces each counter onto its OWN cache line
};

const long ITERS = 100000000;

template <typename T>
double runTest(T& counters) {
    auto t0 = std::chrono::high_resolution_clock::now();
    std::thread ta([&] {
        for (long i = 0; i < ITERS; i++) counters.counterA++;
    });
    std::thread tb([&] {
        for (long i = 0; i < ITERS; i++) counters.counterB++;
    });
    ta.join();
    tb.join();
    auto t1 = std::chrono::high_resolution_clock::now();
    return std::chrono::duration<double, std::milli>(t1 - t0).count();
}

int main() {
    CountersBad bad;
    CountersGood good;
    printf("false sharing: %.2f ms\n", runTest(bad));
    printf("padded:        %.2f ms\n", runTest(good));
}
false sharing: 612.40 ms
padded:        78.90 ms

alignas(64) forces each counter to start at its own 64-byte boundary, so counterA and counterB land in different cache lines. The threads stop fighting over the same line, and the padded version runs about 7.8x faster — despite doing the exact same increments.

Common mistake False sharing does not produce a wrong answer — both versions above compute correct totals. It only shows up as unexplained slowness under multithreading, which makes it easy to miss with normal debugging or even with a correctness-focused profiler. Suspect it whenever multithreaded code scales worse than expected as you add threads, even though single-threaded correctness is fine.

9. SIMD: One Instruction, Many Numbers

Everything so far has been about not wasting time waiting for memory. SIMD attacks a different half of the problem: doing more actual computation per instruction.

SIMD stands for Single Instruction, Multiple Data. A normal ("scalar") instruction operates on one value at a time — one add instruction adds two floats and produces one float. A SIMD instruction packs several values into one wide register and performs the same operation on all of them with a single instruction.

scalar (one lane): a[0] + b[0] = r[0] 1 instruction, 1 result SIMD (4 lanes, e.g. a 128-bit register): +--------+--------+--------+--------+ | a[0] | a[1] | a[2] | a[3] | register A +--------+--------+--------+--------+ +--------+--------+--------+--------+ | b[0] | b[1] | b[2] | b[3] | register B +--------+--------+--------+--------+ + + + + ONE add instruction +--------+--------+--------+--------+ | r[0] | r[1] | r[2] | r[3] | register R +--------+--------+--------+--------+

Each slot in the wide register is called a lane. The register width determines how many lanes you get:

register width floats per lane group instruction set 128-bit 4 SSE / NEON 256-bit 8 AVX / AVX2 512-bit 16 AVX-512

SSE (128-bit) is available on essentially every x86 CPU built this century; AVX and AVX2 (256-bit) are common on modern desktop and console CPUs; AVX-512 (512-bit) shows up on some server and high-end desktop chips. ARM's equivalent is NEON (128-bit, 4 floats), used on mobile phones, Apple Silicon, and the Nintendo Switch.

SIMD is a different kind of parallelism from multithreading. Multithreading runs separate instruction streams on separate cores. SIMD runs one instruction stream, but each instruction does 4 or 8 (or 16) times the work. The two stack: a multithreaded, SIMD-vectorized loop gets both kinds of speedup at once, which is exactly what Section 13's Burst example does.

10. Scalar vs SIMD: Adding Two Float Arrays

The simplest possible SIMD example is adding two arrays of floats element by element.

void addArrays(const float* a, const float* b, float* result, int n) {
    for (int i = 0; i < n; i++) {
        result[i] = a[i] + b[i];
    }
}
#include <immintrin.h>   // SSE / AVX intrinsics

void addArraysSIMD(const float* a, const float* b, float* result, int n) {
    int i = 0;
    for (; i + 4 <= n; i += 4) {
        __m128 va = _mm_loadu_ps(&a[i]);
        __m128 vb = _mm_loadu_ps(&b[i]);
        __m128 vr = _mm_add_ps(va, vb);
        _mm_storeu_ps(&result[i], vr);
    }
    for (; i < n; i++) {           // leftover elements, n not divisible by 4
        result[i] = a[i] + b[i];
    }
}

What each intrinsic call does

Measured on a typical desktop CPU, adding two arrays of 20,000,000 floats:

scalar: 21.85 ms
simd:    7.42 ms

About 2.9x faster, not the full 4x you might expect from 4 lanes. The reason is that this particular loop is memory-bandwidth bound: for every add, the CPU has to bring in two floats from RAM and write one back out. Making the arithmetic itself 4x faster does not help once the bottleneck is how fast RAM can feed data in and take results out — the CPU spends most of its time waiting either way. Compute-heavy SIMD code — many multiplications and additions per value already sitting in a register, like a matrix transform or a physics inner loop — gets much closer to the full 4x-8x speedup, because there the bottleneck really is arithmetic throughput, not memory.

Tip Hand-written intrinsics are a last resort. Profile first, let the compiler auto-vectorize (next section), and only reach for explicit intrinsics on the one specific hot loop that actually needs it and that the compiler is not vectorizing well on its own.

11. Auto-Vectorization: Writing Loops the Compiler Can Speed Up

You rarely need to write intrinsics by hand. Modern compilers — GCC, Clang, and MSVC — can turn a plain scalar loop into SIMD instructions automatically, at optimization level -O2 or -O3, with no intrinsics in your source at all.

// compiled with -O3 -march=native -- no intrinsics written by hand
void addArrays(const float* a, const float* b, float* result, int n) {
    for (int i = 0; i < n; i++) {
        result[i] = a[i] + b[i];
    }
}

Compiled with -O3 -march=native, this ordinary-looking loop is typically turned into SIMD instructions by the compiler on its own, ending up close in speed to the hand-written intrinsics version from the previous section.

Signs a loop will not auto-vectorize

void addArraysFast(const float* __restrict a,
                    const float* __restrict b,
                    float* __restrict result, int n) {
    for (int i = 0; i < n; i++) {
        result[i] = a[i] + b[i];
    }
}

__restrict tells the compiler these three pointers never overlap, so it is free to load, add, and store in batches of 4 or 8 without worrying that writing to result[i] might change a later a[i] or b[i]. Simple loops with no branches, a fixed stride, and no aliasing ambiguity are exactly the loops that vectorize best — which is also why keeping hot loops simple is good advice on its own, separate from readability.

Tip To check whether a loop actually vectorized, compilers can report it directly — GCC and Clang support -fopt-info-vec and -fopt-info-vec-missed. For a quick visual check without setting up flags locally, pasting a function into an online compiler explorer and reading the generated assembly for SIMD instructions (names containing ps or pd on x86) works well too.

12. Where This Shows Up in Real Games

Cache-friendly layout and SIMD show up constantly in the systems that run every single frame over large numbers of similar things:

Notice the common shape: a large array of mostly-independent elements, doing the same small amount of math each frame. That shape is exactly what this whole chapter has been optimizing for — it is not a coincidence that engines organize hot data this way on purpose.

13. Getting SIMD in C# Without Intrinsics: Burst and the Job System

Everything in this chapter so far has been C++. C# on its own does not reliably auto-vectorize, and plain C# does not give you hand-written intrinsics as naturally as C++ does. Unity solves this with two pieces that work together: the Burst compiler and the Job System.

Burst takes a restricted, high-performance subset of C# — value types, NativeArray, no managed classes or garbage collection inside the compiled region — and compiles it straight to optimized native machine code using the same LLVM backend that powers Clang's -O3, including auto-vectorization. A method marked [BurstCompile] is not interpreted or JIT-compiled like ordinary C# — it is compiled ahead of time to real SIMD-capable machine code.

The Job System schedules work across worker threads on multiple cores, safely — Unity's safety checks catch unsynchronized overlapping memory access in the editor before it becomes a hard-to-reproduce bug. IJobParallelFor specifically splits a loop over a NativeArray into batches and runs those batches across however many worker threads are available.

using Unity.Burst;
using Unity.Collections;
using Unity.Jobs;
using Unity.Mathematics;

[BurstCompile]
public struct UpdatePositionsJob : IJobParallelFor
{
    [ReadOnly] public NativeArray<float3> velocities;
    public NativeArray<float3> positions;
    public float deltaTime;

    public void Execute(int index)
    {
        positions[index] = positions[index] + velocities[index] * deltaTime;
    }
}
NativeArray<float3> positions = new NativeArray<float3>(1000000, Allocator.Persistent);
NativeArray<float3> velocities = new NativeArray<float3>(1000000, Allocator.Persistent);
// ... fill both with starting data ...

var job = new UpdatePositionsJob {
    positions = positions,
    velocities = velocities,
    deltaTime = Time.deltaTime
};

JobHandle handle = job.Schedule(positions.Length, 64); // batch size 64
handle.Complete();

Reading the job code

Measured updating 1,000,000 particles' positions from their velocities, once per frame:

plain C# foreach, List<ParticleAoS>, no Burst:   38.70 ms
Burst IJobParallelFor, NativeArray<float3>:        1.15 ms

The 33x gap here is bigger than the 2.9x from Section 10's pure-SIMD C++ example, because three effects stack together: SIMD auto-vectorization inside each thread, multiple worker threads running in parallel across cores, and skipping the managed-object and garbage-collector overhead that a plain List<ParticleAoS> foreach loop carries. None of it works, though, without the data already being laid out as flat NativeArrays — the SoA-shaped storage from Sections 5 and 6, now paying off directly in C#.

Common mistake Burst has real restrictions: no managed classes, no string, no exceptions, no reference types inside a [BurstCompile] job — only blittable value types (types with a fixed, C-like memory layout) and Unity's Native* containers. That is a bigger topic than this chapter covers, but expect the compiler to reject a job that reaches for ordinary managed C# features.

14. Glossary

15. Exercises

Exercise 1 The functions below sum a square matrix of floats, stored row-major (the same layout as Section 4), one column at a time.
double columnSum(const std::vector<float>& matrix, int N, int col) {
    double total = 0.0;
    for (int row = 0; row < N; row++) {
        total += matrix[row * N + col];
    }
    return total;
}

double sumAllColumns(const std::vector<float>& matrix, int N) {
    double grand = 0.0;
    for (int col = 0; col < N; col++) {
        grand += columnSum(matrix, N, col);
    }
    return grand;
}
1. Is sumAllColumns cache-friendly? Explain why, in terms of cache lines.
2. Rewrite it so it computes the exact same grand total, but visits memory in cache-friendly order.
Show answer

No — columnSum's inner loop advances by row, which jumps N floats (one entire row) between accesses. For a large N this is far bigger than a cache line, so almost every access is a fresh cache line fetch, and sumAllColumns repeats this for every column.

double sumAllColumnsFast(const std::vector<float>& matrix, int N) {
    double grand = 0.0;
    for (int row = 0; row < N; row++) {
        for (int col = 0; col < N; col++) {
            grand += matrix[row * N + col];
        }
    }
    return grand;
}

This visits every element exactly once, in the same order the matrix is laid out in memory — row by row. Every cache line fetched is fully used (all 16 of its floats get added) before moving to the next one, instead of being fetched once per column and reused only 1/16th as much.

Exercise 2 An AoS Enemy array holds 100,000 enemies. Every frame, countAlive checks how many are still alive — but it only ever reads health.
struct Enemy {
    float x, y;
    float health;
    float armor;
    char name[32];
    int aiState;
    float attackCooldown;
};

int countAlive(const std::vector<Enemy>& enemies) {
    int alive = 0;
    for (const auto& e : enemies) {
        if (e.health > 0.0f) alive++;
    }
    return alive;
}
Redesign this as SoA for the fields countAlive actually needs, and write the SoA version of the function. Explain why it is more cache-friendly.
Show answer
struct EnemySoA {
    std::vector<float> x, y;
    std::vector<float> health;    // countAlive only needs this array
    std::vector<float> armor;
    std::vector<std::string> name;
    std::vector<int> aiState;
    std::vector<float> attackCooldown;
};

int countAliveSoA(const EnemySoA& enemies) {
    int alive = 0;
    for (float h : enemies.health) {
        if (h > 0.0f) alive++;
    }
    return alive;
}

The AoS Enemy struct is well over 50 bytes — name[32] alone is half a cache line. Every call to countAlive drags position, armor, the whole name buffer, AI state, and cooldown into cache for every single enemy, even though none of it is read. countAliveSoA touches only the health array, so 16 enemies' worth of health values arrive per cache line and every one of them gets used.

Exercise 3 Write a Burst-compiled IJobParallelFor job named ScaleJob that multiplies every element of a NativeArray<float> by a constant factor, plus the code to schedule it. Then explain why NativeArray<float> and [BurstCompile] both matter for getting SIMD speed here.
Show answer
[BurstCompile]
public struct ScaleJob : IJobParallelFor
{
    public NativeArray<float> values;
    public float factor;

    public void Execute(int index)
    {
        values[index] = values[index] * factor;
    }
}
var job = new ScaleJob { values = values, factor = 2.0f };
JobHandle handle = job.Schedule(values.Length, 64);
handle.Complete();

NativeArray<float> is a flat, unmanaged, contiguous buffer — the same layout a C array has, and exactly the layout SIMD hardware needs to load 4 or 8 lanes at once. A managed List<float> is a .NET object the garbage collector can move at any time, so Burst cannot compile against it at all. [BurstCompile] tells Unity to compile Execute ahead of time with the Burst/LLVM backend instead of running it as interpreted or JIT-compiled managed C#, which is what makes auto-vectorization possible — the same effect as -O3 in the C++ sections.

← Back to all chapters