7.8 Real-Time Rendering Techniques

Every technique in this chapter exists because of one hard rule: a game has to draw a new image many times a second, forever, without ever missing its deadline. A movie renderer can spend ten minutes on one frame of a Pixar film. A game engine gets a few milliseconds. This chapter is about the tricks shipping engines — Unreal, Unity's HDRP, and the in-house engines at studios like HoYoverse and Riot — use to fake shadows, smooth edges, indirect light, and reflections well enough, fast enough, that a player never notices the shortcut. Earlier chapters covered the rendering pipeline and shaders in general terms (vertices in, pixels out). This chapter is about the specific, named techniques built on top of that pipeline that make a real-time image look convincing.

1. What "Real-Time" Means: the Frame Budget

A game running at 60 frames per second (FPS, frames per second — how many complete images are shown every second) has to finish a new frame every 16.6 milliseconds (ms, one thousandth of a second). If anything in that frame — game logic, physics, animation, rendering — takes longer, the frame rate drops and the game visibly stutters. This time limit is called the frame budget.

#include <cstdio>

int main() {
    double target_fps = 60.0;
    double frame_budget_ms = 1000.0 / target_fps;
    printf("Target FPS: %.1f\n", target_fps);
    printf("Frame budget: %.3f ms\n", frame_budget_ms);
    return 0;
}

Output:

Target FPS: 60.0
Frame budget: 16.667 ms

16.667 ms is the entire budget — game logic, physics, animation, audio, AI, and rendering all have to fit inside it, and rendering itself is often only a fraction of that. A game targeting 30 FPS gets roughly double the budget (33.3 ms); one targeting 120 FPS for a competitive shooter gets less than half (8.3 ms). Every technique in this chapter is a trade: how much realism can you buy for how many milliseconds.

Recall the basic pipeline shape from earlier chapters: the CPU decides what to draw and submits commands, the GPU (Graphics Processing Unit) turns triangles into pixels through vertex shading, rasterization, and fragment shading, and the result lands on screen.

Everything from here on is about what happens inside that GPU column: how to get shadows, smooth edges, indirect light, and reflections out of it without blowing the budget.

2. Shadow Maps: Depth From the Light's Point of View

A shadow is really one question, asked for every pixel on screen: "is anything between this point and the light?" Checking that directly, for every light and every pixel, by testing against every other object in the scene, is far too slow for a frame budget. The standard trick is called a shadow map, and it turns that expensive question into a single number comparison.

The idea is a two-pass render. In the first pass, you place a camera at the light and render the scene from there, but you only care about depth (distance from the light), not color. Whatever the light "sees" first at each point of its view is the closest surface to the light along that direction — call that a depth buffer (a grid of distances, one per pixel, also called a z-buffer), and when it is used this way it is the shadow map. In the second pass, you render normally from the camera, and for every pixel you project that pixel's position into the light's view and compare its distance to what the shadow map already recorded. If the pixel is farther from the light than the shadow map says the nearest surface is, something else is blocking the light — the pixel is in shadow.

Here is that comparison as a tiny simulation. Treat the shadow map as a plain array (the same kind of array from earlier chapters) — one row of depth values as seen from the light, with a box sitting in the middle of its view (texels 3 and 4 are close, meaning something is nearby there):

#include <cstdio>

// Depth values captured from the LIGHT's point of view (a tiny shadow map).
// Index = which texel; value = distance from the light to the surface it sees there.
float shadow_map[8] = { 5.0f, 5.0f, 5.0f, 2.0f, 2.0f, 5.0f, 5.0f, 5.0f };

// texel: which shadow-map texel this point projects onto
// dist_from_light: the ACTUAL distance from the light to this surface point
bool is_lit(int texel, float dist_from_light) {
    float bias = 0.01f; // a small offset explained below
    float closest = shadow_map[texel];
    return dist_from_light <= closest + bias;
}

int main() {
    // Point A: on the floor at texel 3, and it IS the closest surface there (2.0)
    printf("Point A lit? %s\n", is_lit(3, 2.0f) ? "yes" : "no");
    // Point B: on the floor under the box, texel 3, but the box (2.0) is closer
    printf("Point B lit? %s\n", is_lit(3, 4.5f) ? "yes" : "no");
    return 0;
}

Output:

Point A lit? yes
Point B lit? no

Point A's actual distance from the light (2.0) matches what the shadow map recorded as the closest surface at that texel, so nothing is blocking it: lit. Point B is farther from the light (4.5) than the shadow map's recorded closest surface (2.0) — something (the box) is between it and the light, so it is in shadow. That one comparison, repeated per pixel per light, is the entire algorithm real engines run.

Shadow acne and the bias

Notice the bias variable added before the comparison. Without it, a surface can incorrectly shadow itself. A shadow map only has limited resolution and limited depth precision, so a flat floor lit at a grazing angle can round its own depth up or down slightly differently in the two passes, causing some pixels to fail the comparison against their own true depth. The result is a speckled, noisy pattern of shadow across a surface that should be fully lit, called shadow acne. Pushing the comparison point slightly toward the light (the bias) gives the surface a little slack so it stops failing against itself.

3. Cascaded Shadow Maps: Shadows Across Huge Scenes

A single shadow map has a fixed number of texels, say 2048x2048. If it has to cover a view distance of hundreds of meters (a large open-world level), each texel ends up covering a big chunk of world space, and shadows near the camera — right where the player is looking closely — turn out blocky. If you instead size the shadow map tightly around just the nearby area, distant objects lose their shadows entirely once they leave that tight area.

Cascaded shadow maps (CSM) solve this by not using one shadow map at all. The camera's view frustum (the pyramid-shaped volume it can see) is sliced into several distance ranges called cascades — near, middle, far — and each cascade gets its own shadow map, sized to just that range. The near cascade covers a small area with high texel density (sharp shadows close to the camera, where it matters most); the far cascade covers a huge area with low texel density (blockier shadows far away, where the player won't notice).

A common way to pick the split distances blends two simple schemes: a uniform split (equal-sized distance ranges, which wastes resolution far away) and a logarithmic split (ranges that grow with distance, matching how perspective makes distant things look smaller anyway). Real engines blend the two with a tunable weight, lambda:

#include <cstdio>
#include <cmath>

// Compute 4 cascade split distances between the near and far planes,
// blending a logarithmic split and a uniform split.
void compute_splits(float near_p, float far_p, int count, float lambda, float out[]) {
    for (int i = 1; i <= count; ++i) {
        float p = (float)i / (float)count;
        float log_split = near_p * powf(far_p / near_p, p);
        float uniform_split = near_p + (far_p - near_p) * p;
        out[i - 1] = lambda * log_split + (1.0f - lambda) * uniform_split;
    }
}

// Which cascade should shade a pixel at this view-space depth?
int pick_cascade(float view_depth, float splits[], int count) {
    for (int i = 0; i < count; ++i) {
        if (view_depth <= splits[i]) return i;
    }
    return count - 1;
}

int main() {
    float splits[4];
    compute_splits(0.1f, 200.0f, 4, 0.5f, splits);
    for (int i = 0; i < 4; ++i)
        printf("Cascade %d ends at %.2f units\n", i, splits[i]);

    printf("A pixel at depth 12.0 uses cascade %d\n", pick_cascade(12.0f, splits, 4));
    printf("A pixel at depth 150.0 uses cascade %d\n", pick_cascade(150.0f, splits, 4));
    return 0;
}

Output:

Cascade 0 ends at 25.37 units
Cascade 1 ends at 52.26 units
Cascade 2 ends at 89.97 units
Cascade 3 ends at 200.00 units
A pixel at depth 12.0 uses cascade 0
A pixel at depth 150.0 uses cascade 3

Notice the cascades get much wider as distance grows: the first cascade covers only about 25 units, but the last one covers over 100 units on its own. A nearby pixel (depth 12.0) lands in the tight, high-resolution cascade 0; a distant pixel (depth 150.0) lands in the loose, low-resolution cascade 3. Each cascade needs its own full shadow-map render pass, so 4 cascades roughly means 4 shadow passes per light — cascades trade extra render passes for shadow quality that stays sharp near the camera at any view distance.

4. Soft Shadow Edges: PCF (Percentage-Closer Filtering)

A single shadow-map lookup gives a hard yes/no answer per pixel, which produces a razor-sharp shadow edge — itself a form of aliasing (Section 5 covers this in general), since a real shadow edge softens gradually as you move away from the object casting it. PCF (percentage-closer filtering) fixes this cheaply: instead of testing one texel of the shadow map, test several texels in a small area around the point and average how many of them say "lit." The result is a fraction between 0 (fully shadowed) and 1 (fully lit) instead of a hard binary answer, which reads as a soft, gradient edge.

#include <cstdio>

// Shadow map depths as seen from the light. 2.0 = top of a box, 5.0 = the floor beyond it.
float shadow_map[7][7] = {
    {5,5,5,5,5,5,5},
    {5,5,5,5,5,5,5},
    {5,5,2,2,2,5,5},
    {5,5,2,2,2,5,5},
    {5,5,2,2,2,5,5},
    {5,5,5,5,5,5,5},
    {5,5,5,5,5,5,5}
};

// Percentage-Closer Filtering: sample a 3x3 grid around the texel and
// average how many samples say "lit" instead of trusting a single texel.
float pcf_shadow(int cx, int cy, float dist_from_light) {
    float bias = 0.01f;
    int lit_count = 0;
    for (int dy = -1; dy <= 1; ++dy)
        for (int dx = -1; dx <= 1; ++dx)
            if (dist_from_light <= shadow_map[cy + dy][cx + dx] + bias)
                lit_count++;
    return (float)lit_count / 9.0f;
}

int main() {
    float d = 2.4f; // actual distance from light to this floor point
    printf("Deep inside the box's shadow  (3,3): %.3f\n", pcf_shadow(3, 3, d));
    printf("Right at the shadow's edge    (2,2): %.3f\n", pcf_shadow(2, 2, d));
    printf("Mostly outside the shadow     (1,1): %.3f\n", pcf_shadow(1, 1, d));
    return 0;
}

Output:

Deep inside the box's shadow  (3,3): 0.000
Right at the shadow's edge    (2,2): 0.556
Mostly outside the shadow     (1,1): 0.889

Deep inside the box's shadow, all 9 sampled texels agree it's occluded: 0.000, fully dark. Right at the boundary of the box, the 3x3 sample straddles both occluded and open texels: 0.556, a partial gray value — this is the soft edge. Mostly outside the shadow, only one sampled texel still disagrees: 0.889, nearly fully lit. Real engines usually use more samples (16 or more, often arranged in a rotated or Poisson-disk pattern instead of a plain grid, to hide repeating patterns) and can grow the sample radius based on distance to the object casting the shadow, called PCSS (percentage-closer soft shadows), which makes shadows closer to their caster sharper and shadows farther from their caster softer — matching how real shadows behave under an area light. The core idea stays the one shown above: average several tests instead of trusting one.

5. Why Edges Look Jagged: Aliasing and Point Sampling

A triangle's edge, mathematically, is a perfectly straight line. But a screen is a fixed grid of square pixels, and the rasterizer (the fixed GPU hardware stage that decides which pixels a triangle covers) usually makes its decision by checking a single point — normally the pixel's exact center — and asking "is this one point inside the triangle or not?" That binary decision, repeated pixel by pixel, turns a smooth diagonal or curved edge into a staircase. This general problem — a continuous shape represented by a fixed grid of discrete samples losing information — is called aliasing, and it is the reason diagonal and curved edges in games look "jagged" without further work.

#include <cstdio>

// A triangle's edge, simplified to the line x + y = 8.
// A pixel counts as "inside" if its CENTER point satisfies the test.
bool inside_triangle(int x, int y) {
    float cx = x + 0.5f;
    float cy = y + 0.5f;
    return (cx + cy) < 8.0f;
}

int main() {
    for (int y = 0; y < 8; ++y) {
        for (int x = 0; x < 8; ++x) {
            putchar(inside_triangle(x, y) ? '#' : '.');
        }
        putchar('\n');
    }
    return 0;
}

Output:

#######.
######..
#####...
####....
###.....
##......
#.......
........

The math boundary is a single straight diagonal line. What actually got drawn is a staircase, because each pixel only ever asked "is my one center point inside or outside," with no notion of "I am 30% covered." Every anti-aliasing (AA) technique in the next three sections is a different answer to the same underlying problem: get a better answer than a single yes/no sample per pixel, without paying the full cost of, say, rendering the image at 4x or 16x resolution and shrinking it down afterward (called supersampling or SSAA, which works but is expensive because it reruns the entire pipeline, shading included, many times per final pixel).

6. MSAA: Multi-Sample Anti-Aliasing

MSAA (multi-sample anti-aliasing) improves on the single-center-sample test from Section 5 without paying the full cost of supersampling. The trick: test coverage (is this sub-pixel point inside the triangle?) at several fixed positions inside each pixel, but only run the expensive fragment shader (the actual color/lighting math) once per pixel per triangle. The final pixel color then blends based on what fraction of those sample points were covered.

#include <cstdio>

bool inside_triangle(float x, float y) {
    return (x + y) < 7.0f;
}

// 4x MSAA: test 4 sub-pixel sample points instead of just the pixel center.
float msaa4_coverage(int px, int py) {
    float offsets[4][2] = {
        {0.25f, 0.25f}, {0.75f, 0.25f},
        {0.25f, 0.75f}, {0.75f, 0.75f}
    };
    int covered = 0;
    for (int i = 0; i < 4; ++i) {
        float sx = px + offsets[i][0];
        float sy = py + offsets[i][1];
        if (inside_triangle(sx, sy)) covered++;
    }
    return covered / 4.0f;
}

int main() {
    printf("Pixel (2,3) coverage: %.2f\n", msaa4_coverage(2, 3));
    printf("Pixel (3,3) coverage: %.2f\n", msaa4_coverage(3, 3));
    printf("Pixel (4,3) coverage: %.2f\n", msaa4_coverage(4, 3));
    return 0;
}

Output:

Pixel (2,3) coverage: 1.00
Pixel (3,3) coverage: 0.25
Pixel (4,3) coverage: 0.00

Pixel (2,3) sits entirely inside the triangle: all 4 samples agree, coverage 1.00, full color. Pixel (4,3) sits entirely outside: coverage 0.00. Pixel (3,3) straddles the edge: only 1 of 4 samples is inside, coverage 0.25, so the final pixel blends 25% triangle color with whatever is behind it — a soft, partially-covered edge instead of a hard jump. Because the expensive fragment shader still runs only once per pixel (not 4 times), MSAA is much cheaper than full 4x supersampling while still smoothing geometric edges.

7. FXAA: Fast Approximate Post-Process Anti-Aliasing

MSAA needs access to geometry and coverage information while the scene is being rasterized. FXAA (fast approximate anti-aliasing) takes a completely different approach: it runs after the entire image is already rendered, as a post-processing pass (an image-processing step applied to the final 2D picture, with no knowledge of the 3D scene that produced it). It scans the finished image for sharp brightness jumps — high-contrast edges, which usually mean a jagged geometric edge — and blends those pixels with their neighbors to soften them.

#include <cstdio>
#include <cmath>

// A row of pixel brightness values (0..1), with a hard edge in the middle.
float pixels[8] = {0.9f, 0.9f, 0.9f, 0.9f, 0.1f, 0.1f, 0.1f, 0.1f};

int main() {
    float smoothed[8];
    for (int i = 0; i < 8; ++i) smoothed[i] = pixels[i];

    for (int i = 1; i < 7; ++i) {
        float left = pixels[i - 1];
        float right = pixels[i + 1];
        float contrast = fabsf(left - right);
        if (contrast > 0.3f) {
            // High contrast: likely an edge. Blend this pixel with its neighbors.
            smoothed[i] = (left + pixels[i] + right) / 3.0f;
        }
    }

    for (int i = 0; i < 8; ++i) printf("%.2f ", smoothed[i]);
    printf("\n");
    return 0;
}

Output:

0.90 0.90 0.90 0.63 0.37 0.10 0.10 0.10 

The original row jumps sharply from 0.9 to 0.1 in one step — a hard edge. The high-contrast check catches pixels near that jump and blends them with their neighbors, turning the sudden step into a short ramp: 0.9, 0.63, 0.37, 0.1. The edge is now a couple of pixels of gradient instead of one sharp line, which reads as smooth from normal viewing distance. FXAA is popular because it is very cheap (one pass over the final image, no extra geometry rendering) and works with any renderer, but because it only looks at 2D contrast, it cannot tell a real geometric edge apart from a high-contrast texture detail, like a painted line or small text.

8. TAA: Temporal Anti-Aliasing

MSAA spends extra samples within one frame. TAA (temporal anti-aliasing) spends extra samples across frames instead: each frame, the sample position is shifted slightly (called jitter, a small, changing sub-pixel offset), and the new, jittered frame is blended with an accumulated history of previous frames. Over several frames, this converges toward the same answer a multi-sample test would have given in a single frame — but spread out over time, at almost no extra cost per frame.

Reusing a previous frame's pixel requires knowing where that surface point is now, since the camera or the object may have moved. Engines compute a velocity buffer (also called a motion vector buffer): for every pixel, how far did this surface point move on screen since last frame? That vector is used to reproject — look up where this same surface point was in the previous frame's image — before blending.

Here is the same triangle edge from the MSAA example, but sampled one jittered point per frame instead of four points in one frame, and blended into a running history using a fixed blend weight:

#include <cstdio>

bool inside_triangle(float x, float y) {
    return (x + y) < 7.0f;
}

int main() {
    // TAA jitters the sample point to a different sub-pixel offset every
    // frame, then blends the new sample into the accumulated history.
    float jitter_x[4] = {0.25f, 0.75f, 0.25f, 0.75f};
    float jitter_y[4] = {0.25f, 0.25f, 0.75f, 0.75f};
    int px = 3, py = 3;

    float history = 0.5f; // first guess, no history yet
    float alpha = 0.25f;  // how much weight the new frame gets

    for (int frame = 0; frame < 4; ++frame) {
        float sample = inside_triangle(px + jitter_x[frame], py + jitter_y[frame]) ? 1.0f : 0.0f;
        history = history * (1.0f - alpha) + sample * alpha;
        printf("Frame %d: jittered sample=%.1f  accumulated=%.4f\n", frame, sample, history);
    }
    return 0;
}

Output:

Frame 0: jittered sample=1.0  accumulated=0.6250
Frame 1: jittered sample=0.0  accumulated=0.4688
Frame 2: jittered sample=0.0  accumulated=0.3516
Frame 3: jittered sample=0.0  accumulated=0.2637

These are the exact same four jitter offsets used as the four MSAA sample points in Section 6, where pixel (3,3) came out to a true coverage of 0.25. Here, spread one-per-frame across four frames and blended, the accumulated value is drifting down toward that same 0.25 — TAA reconstructs the same kind of answer as MSAA, using time instead of extra samples in a single frame, at a fraction of the per-frame cost.

9. Forward Shading vs. Deferred Shading

Lighting math needs to happen somewhere in the pipeline: for every visible pixel, combine its surface material with every light that reaches it. There are two very different ways to organize that work.

Forward shading is the direct approach: for each object, for each light that might affect it, compute the lighting right there in the object's fragment shader, one triangle at a time. It is simple and works naturally with transparency and MSAA, but it has a problem — overdraw (shading the same screen pixel more than once, because a later, closer object gets drawn on top of an earlier one that already paid the full shading cost) multiplies badly with a large light count, since every overdrawn pixel pays for every light again.

Deferred shading splits the work into two passes instead. The first pass, the geometry pass, draws every object but does not compute any lighting yet — it just writes each pixel's material information (surface color, normal direction, depth, roughness) into a set of full-screen textures called the G-buffer (geometry buffer). The second pass, the lighting pass, runs once per final screen pixel, reads the G-buffer, and loops over the lights that reach that pixel. Lighting math only ever runs on the pixels that actually end up visible on screen, not on every overdrawn layer underneath them.

A small simulation makes the difference concrete. Say a scene has 50 objects, each averaging 2000 pixels of screen coverage (with some overlap between objects), 40 lights, and the final visible image is 80,000 pixels:

#include <cstdio>

int main() {
    int objects = 50;
    int pixels_per_object = 2000; // average screen coverage, including overlap
    int lights = 40;              // e.g. many small point lights in a level

    // Forward: naively shades EVERY light against EVERY pixel of EVERY
    // object, even pixels that later get drawn over by something closer.
    long forward_ops = (long)objects * pixels_per_object * lights;

    // Deferred: pay the material write once per pixel (geometry pass),
    // then pay the lighting cost once per pixel per light, on the FINAL
    // visible pixels only - overdraw never reaches the lighting pass.
    long screen_pixels = 80000;
    long gbuffer_ops = screen_pixels;
    long lighting_ops = screen_pixels * lights;
    long deferred_ops = gbuffer_ops + lighting_ops;

    printf("Forward shading operations:  %ld\n", forward_ops);
    printf("Deferred shading operations: %ld\n", deferred_ops);
    return 0;
}

Output:

Forward shading operations:  4000000
Deferred shading operations: 3280000

Notice objects x pixels_per_object (100,000) is larger than the final screen_pixels (80,000) — that 1.25x gap is overdraw: some screen pixels got shaded by more than one overlapping object in the forward version. Deferred shading pays the material cost exactly once per final pixel and never re-pays the lighting cost for pixels that end up hidden. The gap between forward and deferred grows fast as the light count grows, which is why deferred shading became popular specifically for scenes with many small dynamic lights.

10. Approximating Global Illumination

Global illumination (GI) means light that bounces more than once before reaching your eye — a white wall lit by direct sunlight also lights up the room around it, a red rug tints the wall next to it slightly red. Simulating this exactly means tracing how light bounces around an entire scene, which is far too slow for a frame budget. Real-time engines instead use a handful of targeted approximations, each covering a different piece of what full GI would give you.

Light probes: pre-baked indirect light at a few points

A light probe is a point in the level where, ahead of time (usually when the level is built, not while the game is running), the engine calculates what indirect light looks like at that exact spot — light bounced from walls, floors, and other static geometry. Probes are placed throughout a level, often more densely in visually important areas. At runtime, a moving object (which cannot have its own pre-baked lighting, since it can be anywhere) looks up the nearest probes and blends between them based on distance, to get a cheap estimate of the indirect light hitting it right now.

#include <cstdio>

// Two light probes along a hallway, storing indirect light (simplified to
// one brightness number instead of full RGB for clarity).
float probe_A_light = 0.2f; // near a dark corner
float probe_B_light = 0.8f; // near a bright window

int main() {
    float probe_A_pos = 0.0f;
    float probe_B_pos = 10.0f;
    float object_pos = 7.0f; // a crate somewhere between the two probes

    float t = (object_pos - probe_A_pos) / (probe_B_pos - probe_A_pos);
    float indirect_light = probe_A_light * (1.0f - t) + probe_B_light * t;

    printf("Blend factor t = %.2f\n", t);
    printf("Interpolated indirect light on the crate: %.3f\n", indirect_light);
    return 0;
}

Output:

Blend factor t = 0.70
Interpolated indirect light on the crate: 0.620

The crate sits 70% of the way from the dim probe toward the bright probe, so it picks up 70% of the brightness difference between them: 0.62, brighter than the dark corner but dimmer than right next to the window. Real light probes store full spherical lighting information (light arriving from every direction, not one number), and real engines blend more than two neighboring probes at once, but the underlying idea — sample a few pre-computed points and interpolate — is exactly this.

Screen-space reflections (SSR)

Screen-space reflections reuse information the renderer already has on screen — the color buffer and the depth buffer — to approximate reflections without tracing rays through the full 3D scene. A reflection ray is marched (stepped forward little by little) starting from the reflective surface, and at each step, its current depth is compared against what the depth buffer already recorded at that screen position. When the ray's depth catches up to (or passes) the depth buffer's value, that pixel is treated as the hit point, and its already-rendered color is reused as the reflection.

#include <cstdio>

// Depth buffer for a row of screen pixels the reflection ray crosses.
// (Distance from camera; a crate close to the camera sits at index 4-7.)
float depth_buffer[10] = {5,5,5,5, 2,2,2,2, 5,5};

int main() {
    float ray_depth = 1.0f; // the ray starts close to the camera
    float step = 0.5f;      // and marches away from it, pixel by pixel

    for (int px = 0; px < 10; ++px) {
        printf("pixel %d: ray_depth=%.2f buffer_depth=%.2f", px, ray_depth, depth_buffer[px]);
        if (ray_depth >= depth_buffer[px]) {
            printf("  -> HIT: use color at this pixel as the reflection\n");
            break;
        }
        printf("  (ray is still in front of the surface)\n");
        ray_depth += step;
    }
    return 0;
}

Output:

pixel 0: ray_depth=1.00 buffer_depth=5.00  (ray is still in front of the surface)
pixel 1: ray_depth=1.50 buffer_depth=5.00  (ray is still in front of the surface)
pixel 2: ray_depth=2.00 buffer_depth=5.00  (ray is still in front of the surface)
pixel 3: ray_depth=2.50 buffer_depth=5.00  (ray is still in front of the surface)
pixel 4: ray_depth=3.00 buffer_depth=2.00  -> HIT: use color at this pixel as the reflection

The ray marches forward, staying "in front of" the far wall (depth 5.00) for the first four pixels, then at pixel 4 its own depth (3.00) has caught up to the nearby crate's depth (2.00) — a hit. The color already rendered at pixel 4 gets reused as the reflected color. The catch is right there in the name: screen-space. If the thing that should be reflected is off-screen, or hidden behind something else from the camera's point of view, its color was never rendered in the first place, so the ray march fails to find a hit — engines fall back to a pre-baked reflection probe (similar in spirit to a light probe, but storing a full 360-degree image instead of one brightness value) whenever SSR comes up empty.

Screen-space ambient occlusion (SSAO)

Ambient occlusion (AO) is the soft darkening you see in corners, creases, and anywhere two surfaces meet closely — a crevice gets less ambient light than an open flat surface, because nearby geometry blocks part of the sky/environment light that would otherwise reach it. SSAO approximates this using only the depth buffer: for each pixel, sample nearby pixels' depths, and if a neighbor is noticeably closer to the camera than expected, count it as partially blocking light.

#include <cstdio>

// Scene depth (distance from camera) looking into a narrow crevice.
// The crevice walls (4.0) are close to the camera; the back of the
// crevice (7.0) is farther away and partly boxed in by the walls.
float depth[5][5] = {
    {10,10,10,10,10},
    {10, 4, 4, 4,10},
    {10, 4, 7, 4,10},
    {10, 4, 4, 4,10},
    {10,10,10,10,10}
};

// SSAO: check nearby depth samples. A neighbor that is noticeably CLOSER
// to the camera than this pixel blocks some of the ambient light that
// would otherwise reach it.
float ssao(int cx, int cy) {
    float center = depth[cy][cx];
    int occluders = 0, samples = 0;
    for (int dy = -1; dy <= 1; ++dy) {
        for (int dx = -1; dx <= 1; ++dx) {
            if (dx == 0 && dy == 0) continue;
            float neighbor = depth[cy + dy][cx + dx];
            if (neighbor < center - 0.5f) occluders++;
            samples++;
        }
    }
    return 1.0f - (float)occluders / (float)samples; // 1.0 = open, 0.0 = dark
}

int main() {
    printf("AO on the crevice wall (1,1): %.3f\n", ssao(1, 1));
    printf("AO at the crevice back (2,2): %.3f\n", ssao(2, 2));
    return 0;
}

Output:

AO on the crevice wall (1,1): 1.000
AO at the crevice back (2,2): 0.000

The point on the rim of the crevice (1,1) has no neighbor closer than itself, so AO stays at 1.000, fully open. The point at the back of the crevice (2,2) is surrounded on all 8 sides by the closer crevice walls, so every neighbor counts as an occluder: AO drops to 0.000, fully dark. Multiply this factor into the ambient/indirect light term of your lighting, and corners and creases naturally darken.

11. GPU-Driven Rendering and Culling

Rendering an object the camera cannot even see wastes the whole pipeline on it. Culling means deciding, before doing that expensive work, which objects can be skipped entirely.

The simplest and most common form is frustum culling: test each object's bounding volume (a simple shape, like a sphere or box, that fully contains the real mesh, cheap to test instead of testing every triangle) against the camera's frustum (the pyramid-shaped volume the camera can see, bounded by a near plane, a far plane, and the sides of its field of view). Anything entirely outside the frustum gets skipped before a single vertex of it is ever sent to the GPU.

#include <cstdio>
#include <cmath>

struct Sphere { float x, z, radius; };

// A simplified 2D frustum: camera at the origin looking down +z,
// with a field-of-view half-angle, and near/far clip distances.
bool in_frustum(Sphere s, float near_d, float far_d, float half_angle_rad) {
    if (s.z + s.radius < near_d) return false; // fully behind the near plane
    if (s.z - s.radius > far_d) return false;  // fully beyond the far plane

    // Distance the frustum's side plane is from the view axis at this depth.
    float side_limit = s.z * tanf(half_angle_rad);
    if (fabsf(s.x) - s.radius > side_limit) return false; // outside the side plane
    return true;
}

int main() {
    float near_d = 0.5f, far_d = 100.0f, half_angle = 0.6f; // about 34 degrees

    Sphere crate    = {2.0f, 20.0f, 1.0f};   // near the camera's path
    Sphere far_rock = {5.0f, 200.0f, 3.0f};  // way past the far plane
    Sphere off_side = {90.0f, 20.0f, 1.0f};  // way off to the side

    printf("crate visible?    %s\n", in_frustum(crate, near_d, far_d, half_angle) ? "yes" : "no");
    printf("far_rock visible? %s\n", in_frustum(far_rock, near_d, far_d, half_angle) ? "yes" : "no");
    printf("off_side visible? %s\n", in_frustum(off_side, near_d, far_d, half_angle) ? "yes" : "no");
    return 0;
}

Output:

crate visible?    yes
far_rock visible? no
off_side visible? no

The crate passes all three checks and gets drawn. The rock fails the far-plane check outright — it is simply too far away, regardless of angle. The off-to-the-side object passes the near/far checks but fails the side-plane check — it is within drawing distance, but well outside the camera's field of view, so drawing it would be wasted work.

Frustum culling only catches objects outside the camera's view volume. Occlusion culling catches a different case: an object fully inside the frustum but completely hidden behind something closer — a room behind a wall, for example. A common technique is Hi-Z (hierarchical depth buffer): build a mipmap-like pyramid of the depth buffer (each level storing the farthest depth of a block of pixels from the level below it), so testing whether an object's bounding volume is fully hidden behind existing geometry can be done with a handful of coarse lookups instead of comparing against every pixel it might cover.

Doing all of this culling on the CPU, one object at a time, becomes its own bottleneck once a scene has hundreds of thousands of objects — even a cheap-per-object test adds up, and every draw call the CPU decides to submit costs time crossing over to the GPU. GPU-driven rendering moves the decision itself onto the GPU: a compute shader (a general-purpose GPU program, not tied to one vertex or one pixel) tests all objects' visibility in parallel and writes the surviving ones directly into a buffer of indirect draw calls (draw commands whose parameters, like which objects to draw, are read from GPU memory instead of being specified by the CPU ahead of time). The CPU only has to kick off one dispatch; the GPU decides what to actually draw and then draws it, with far less CPU involvement per object.

12. Putting It Together: a Modern Frame, Pass by Pass

None of these techniques run in isolation. A typical frame in a modern deferred renderer runs its passes in a specific order, because each pass usually depends on data an earlier pass produced.

Reading this left to right: the shadow pass renders each light's shadow map first (Sections 2-4), since the main scene pass will need to sample it. A depth prepass often renders just depth for opaque objects before anything else, so later passes can skip shading pixels that end up hidden behind something closer, cutting overdraw even before the G-buffer pass begins. The G-buffer pass (Section 9) writes material data once per visible pixel. SSAO (Section 10) runs next because it only needs the depth/normal data the G-buffer just produced. The lighting pass reads the G-buffer, the shadow maps, and the SSAO result together to produce a lit image. SSR (Section 10) runs after lighting because it needs an already-shaded color buffer to sample reflections from. A separate forward pass handles transparent objects last among the 3D geometry, since they need to blend with everything already drawn behind them. TAA (Section 8) resolves the jittered frame against history near the end, before tonemapping (converting rendered brightness values into displayable colors) and any other post-processing, so those final steps operate on a stable, already-anti-aliased image rather than a jittered one.

Every studio's engine varies this order somewhat, and some passes (like SSR or SSAO) are sometimes skipped entirely on lower-end hardware to save budget. But the shape — shadows and depth first, then material data, then lighting, then screen-space effects that reuse what was already drawn, then temporal resolve, then final color grading — is close to universal across shipping engines.

13. How to Actually Learn This For Real

This chapter gave you the concept and a small worked simulation for each technique. That is enough to recognize these ideas and reason about their trade-offs, but implementing one for real in a working renderer is a different, deeper skill, and it is where the understanding actually sticks. A practical path:

• Pick one technique from this chapter. Not all of them at once — shadow mapping with basic PCF is a reasonable first choice, since it is self-contained and you can see the result immediately.
• Find a primary source. Look for a real conference talk or written course covering how shipped engines actually implement it — the SIGGRAPH "Advances in Real-Time Rendering in Games" course has run for years and covers exactly this kind of production technique; GDC (Game Developers Conference) talks are another good source, and many are freely available. Engine source code is also a primary source: Godot and parts of Unreal Engine are open, and reading their actual shadow-mapping or SSAO code shows you decisions this chapter only had room to summarize.
• Implement the smallest version that works. A toy renderer using a graphics API you already know a little of (or a compute-shader playground) is enough — you do not need a full engine to prove a shadow map or an SSAO pass works.
• Inspect it with a graphics debugger. Tools like RenderDoc (free, cross-platform), PIX (Windows/Xbox), or Nsight Graphics (NVIDIA) let you capture a single frame from any DirectX, Vulkan, or OpenGL application — including games you did not write — and step through it pass by pass, viewing every intermediate texture: the shadow map itself, each G-buffer channel, the raw SSAO output before it's applied. This turns an abstract diagram into something you can actually look at, pixel by pixel.
• Compare against a reference and tune by measuring, not guessing. If you can render (or find) a slower, "obviously correct" version — a brute-force shadow test, a full path-traced reference image — compare your fast approximation against it side by side. Tune parameters like shadow bias or SSAO radius by observing what actually changes, the same debugging discipline from earlier chapters: form a hypothesis, inspect real state (with gdb and AddressSanitizer for CPU code, with a graphics debugger here), and confirm or reject it, rather than changing numbers at random until it looks better.

14. Glossary

• Frame budget — the maximum time available to produce one frame at a target frame rate, e.g. 16.6 ms at 60 FPS.
• Depth buffer (z-buffer) — a grid of distance-from-camera (or from a light) values, one per pixel or texel.
• Shadow map — a depth buffer rendered from a light's point of view, used to test whether a point is blocked from that light.
• Shadow acne — speckled false self-shadowing caused by limited depth precision; fixed with a small bias.
• Peter-panning — a shadow visibly detached from its object, caused by using too much bias.
• Cascaded shadow maps (CSM) — splitting the camera frustum into distance ranges, each with its own shadow map, so shadow resolution stays sharp near the camera across a huge view distance.
• PCF (percentage-closer filtering) — averaging several shadow-map samples around a point to produce a soft shadow edge instead of a hard one.
• PCSS (percentage-closer soft shadows) — PCF with a sample radius that grows based on distance to the object casting the shadow.
• Aliasing — visual artifacts (like jagged edges) caused by representing a continuous shape with a fixed grid of discrete samples.
• Rasterization — the GPU stage that turns a triangle into the pixels it covers on screen.
• Supersampling (SSAA) — rendering at a much higher resolution and shrinking down, the expensive but simple way to reduce aliasing.
• MSAA (multi-sample anti-aliasing) — testing geometric coverage at several sub-pixel points per pixel while shading only once, cheaper than supersampling.
• FXAA (fast approximate anti-aliasing) — a post-process pass that smooths high-contrast edges in the final 2D image.
• TAA (temporal anti-aliasing) — jittering the sample position each frame and blending with reprojected history from previous frames.
• Jitter — a small, changing sub-pixel offset applied to the sample position each frame in TAA.
• Velocity buffer / motion vector — per-pixel data recording how far a surface point moved on screen since the previous frame.
• Reprojection — using a velocity vector to look up where a surface point was in a previous frame's image.
• Ghosting — a trailing smear artifact in TAA caused by blending in history data that no longer matches the current frame.
• Disocclusion — a surface newly revealed on screen because whatever was blocking it moved out of the way.
• Neighborhood clamping — constraining a TAA history sample to the range of colors seen nearby in the current frame, to reduce ghosting.
• Forward shading — computing lighting directly per object per light while rendering geometry.
• Deferred shading — splitting rendering into a geometry pass (writing a G-buffer) and a separate lighting pass that reads it.
• G-buffer — the set of full-screen textures (albedo, normal, depth, roughness/metallic, etc.) written by a deferred renderer's geometry pass.
• Overdraw — shading the same screen pixel more than once because a later, closer object is drawn on top of an earlier one.
• Forward+ / clustered shading — a hybrid that shades in one pass like forward shading, but first sorts lights into screen-space grid cells so each pixel only tests nearby lights.
• Global illumination (GI) — light that bounces more than once before reaching the eye, including indirect/bounced light.
• Light probe — a point in a level where indirect lighting is pre-computed, used to light dynamic objects near that point at runtime.
• Screen-space reflections (SSR) — approximating reflections by marching a ray through the existing depth/color buffers instead of tracing the full 3D scene.
• Screen-space ambient occlusion (SSAO) — approximating ambient occlusion by checking whether nearby depth-buffer samples are closer to the camera than the current pixel.
• Frustum culling — skipping objects whose bounding volume is entirely outside the camera's view volume.
• Occlusion culling — skipping objects that are fully hidden behind other, closer geometry.
• Hi-Z (hierarchical depth buffer) — a mipmap-like pyramid of the depth buffer used to speed up occlusion tests.
• GPU-driven rendering — performing visibility/culling decisions on the GPU itself via compute shaders, instead of on the CPU per object.
• Indirect draw call — a draw command whose parameters are read from GPU memory rather than specified by the CPU ahead of time.
• Graphics debugger — a tool (RenderDoc, PIX, Nsight Graphics) that captures a single rendered frame and lets you inspect every intermediate buffer and draw call.

15. Exercises

#include <cstdio>

float shadow_map[7][7] = {
    {5,5,5,5,5,5,5},
    {5,5,5,5,5,5,5},
    {5,5,2,2,2,5,5},
    {5,5,2,2,2,5,5},
    {5,5,2,2,2,5,5},
    {5,5,5,5,5,5,5},
    {5,5,5,5,5,5,5}
};

float pcf_shadow(int cx, int cy, float dist_from_light) {
    float bias = 0.01f;
    int lit_count = 0;
    for (int dy = -1; dy <= 1; ++dy)
        for (int dx = -1; dx <= 1; ++dx)
            if (dist_from_light <= shadow_map[cy + dy][cx + dx] + bias)
                lit_count++;
    return (float)lit_count / 9.0f;
}

int main() {
    printf("PCF at (4,3): %.3f\n", pcf_shadow(4, 3, 2.4f));
    return 0;
}

PCF at (4,3): 0.333

#include <cstdio>

bool inside_triangle(float x, float y) {
    return (x + y) < 7.0f;
}

int main() {
    float jitter_x[4] = {0.25f, 0.75f, 0.25f, 0.75f};
    float jitter_y[4] = {0.25f, 0.25f, 0.75f, 0.75f};
    int px = 3, py = 3;

    float history = 0.5f;
    float alpha = 0.25f;

    for (int frame = 0; frame < 8; ++frame) {
        int j = frame % 4; // cycle through the same 4 jitter offsets again
        float sample = inside_triangle(px + jitter_x[j], py + jitter_y[j]) ? 1.0f : 0.0f;
        history = history * (1.0f - alpha) + sample * alpha;
        printf("Frame %d: jittered sample=%.1f  accumulated=%.4f\n", frame, sample, history);
    }
    return 0;
}

Frame 0: jittered sample=1.0  accumulated=0.6250
Frame 1: jittered sample=0.0  accumulated=0.4688
Frame 2: jittered sample=0.0  accumulated=0.3516
Frame 3: jittered sample=0.0  accumulated=0.2637
Frame 4: jittered sample=1.0  accumulated=0.4478
Frame 5: jittered sample=0.0  accumulated=0.3358
Frame 6: jittered sample=0.0  accumulated=0.2519
Frame 7: jittered sample=0.0  accumulated=0.1889