Course 4 — Real-Time Rendering & GPU Engineering: Vulkan & Metal in Modern C++

A fully lab-based course: modern C++20 and CMake, the same scenes rendered through Vulkan and Metal, and profiling on every lab — from a cleared swapchain to a shipped mini-engine

A lab-first course that takes someone who already has the mathematics (see the prerequisite below) and turns them into an engineer who can architect, implement, and optimize real-time 3D rendering and GPU compute. Every idea is learned by building it — in modern C++20, built with CMake — running it on real GPUs, capturing it in a profiler, and reconciling the measured frame against what was predicted. The target is fluency for rendering-engineer and GPU-engineer work: the graphics pipeline end to end, engine architecture (resource lifetimes, render graphs, multithreaded command recording), the advanced techniques a modern renderer is made of (shadows, deferred shading, terrain LOD, GPU-driven rendering, point splatting), and the performance engineering that makes it all ship, from battery-powered laptops to desktop GPUs.

Three GPU programming models are treated as one curriculum, because a working rendering engineer needs all three views:

Vulkan (C++20) — the fully explicit model: you own memory, synchronization, and submission. Written against the LunarG SDK; runs on the Mac through MoltenVK and natively on the Linux desktop’s RTX 4090 — the same code on two very different GPUs, which is itself a recurring measurement.
Metal (Swift, then metal-cpp) — the Apple-native model. The API-learning labs are written in Swift, the language of Metal by Tutorials and of Xcode’s first-class GPU tooling; the engine-architecture module then adds a metal-cpp backend, so the C++ engine core drives both APIs — the way cross-platform engines actually ship on Apple GPUs.
CUDA — learned in Course 2, spent here. General-purpose GPU computing on the RTX 4090 — kernels, the memory hierarchy, reductions and scans, streams, occupancy, Nsight profiling — is worked as the 25-exercise CUDA track of Course 2’s 100 exercises to learn modern C++. It is a prerequisite, not a module: the compute mental model (warps, shared memory, bandwidth) is exactly what advanced rendering (GPU culling, sorting, splatting) is built from, and this course assumes it rather than re-teaching it.

Prerequisite: Course 1 — Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence is assumed as mastered — this course applies linear algebra, geometry, and numerical analysis, and never re-teaches them. Where a lab leans on a specific result it links back: Parts I–II (vector spaces, linear maps, inner products — every transform and every lighting dot product), Lesson 42 (floating point, conditioning — depth precision, accumulated error), Lesson 29 (sampling — texture filtering and mipmapping are the sampling theorem), and Course 3 Lab 9.4 theory (2-D signals — convolution post-effects, sRGB). The 3D math itself — coordinate spaces and handedness, homogeneous transforms and perspective projection, quaternions, geometric primitives and bounding volumes — is worked as the Dunn & Parberry exercise set on the Books page. Course 2 is the on-ramp for the C discipline this course extends, for the CUDA compute model it assumes, and for the Python tooling; this course’s Module 0 bridges from C to the C++20 the engine is written in. No graphics experience is assumed.

Note on AI use: As across this site, the lab write-ups are drafted with AI assistance so every lab has the same structure to work from. The substance is mine: every line of engine, shader, and kernel code is written and debugged by me, every capture and benchmark is taken on my own hardware, and every predicted-vs-measured reconciliation is my own. The write-ups specify what to build and how to measure it — they do not contain the solutions. You only learn this by building it.

How each lab is structured

Every lab file follows the same template so it can be worked as a self-contained session:

Goal: what skill/concept the lab builds and why it matters for the craft.
Recommended reading: the specific chapters (from the books below and, where relevant, Courses 1/3) to read before writing code.
Prerequisites: the labs and setup steps this lab assumes are done.
Project & environment setup: the exact CMake targets, repo paths, SDK pieces, and run configurations — so nothing about how to build and run the lab is ambiguous. The rendering/compute code itself is never given (see the AI note above); only the scaffolding is.
Background: the minimum theory, with the key equations and the API concepts involved.
Tasks: numbered implementation steps. Dual-track labs split into Vulkan (C++20) and Metal (Swift) subsections.
Deliverable & expected results: the artifact to produce (screenshot, capture file, benchmark table), the exact files to record and where they go in the repo, and the behavior/numbers to expect.
Profiling & performance: which tool to reach for (Tracy, Xcode GPU capture, RenderDoc, Nsight), what to capture, and the frame/kernel budget to check against.
Analysis & reconciliation: predict by hand, then explain any gap between predicted and measured.
Going further: optional extensions.

Predicted-vs-measured tables leave the Measured cells as “…” — they are filled at the machine, same convention as Course 3’s bench labs.

Hardware & platforms (already on hand)

Apple Silicon Mac (M-series) — the primary development machine and the Apple-GPU target: Metal natively (Xcode, Swift, metal-cpp) and Vulkan through MoltenVK. Its tile-based deferred rendering (TBDR) architecture vs. the 4090’s immediate-mode desktop architecture is a running comparison in Modules 5–6.
Linux desktop (NVIDIA GeForce RTX 4090) — the NVIDIA target: a discrete Ada Lovelace GPU with its own VRAM across PCIe, running native Vulkan on Linux (and hosting Course 2’s CUDA track) — the Linux side of every cross-platform measurement. Power and thermals via nvidia-smi keep efficiency measurable even on a desktop power budget. Development stays on the Mac: CLion’s remote toolchain builds, runs, and debugs on this box over SSH — edit on the laptop, execute on the 4090.
Raspberry Pi 5 (optional) — Vulkan 1.2 through Mesa’s V3DV driver; an optional third port target in the Going-further sections, never required.

No other hardware is needed: every lab runs on the Mac, the Linux box, or both.

Toolchain & software setup

# --- Mac: core toolchain ---
brew install cmake ninja            # CMake ≥ 3.28, Ninja generator
xcode-select --install              # or full Xcode (required for Metal + GPU capture)
# Vulkan: install the LunarG Vulkan SDK for macOS (includes MoltenVK, glslc,
#   validation layers, vulkaninfo). Source setup-env.sh from your shell profile.

# --- Dependencies (fetched by CMake, no manual installs) ---
# GLFW (windowing), GLM (math), VMA (Vulkan Memory Allocator), volk (loader),
# stb (image I/O), tinygltf (meshes), Tracy (frame profiler),
# Google Benchmark (microbenchmarks) — all via FetchContent, pinned in the repo.

# --- Metal ---
# Xcode + Swift for the API-learning labs (MTKView, MSL shaders, GPU capture).
# metal-cpp (single-header C++ bindings) enters in Module 4.

# --- Linux desktop (RTX 4090) ---
# NVIDIA driver, Vulkan SDK / vulkan-tools via apt, RenderDoc (Linux build),
# Nsight Graphics, nvidia-smi for power/thermals.
# Remote workflow: CLion on the Mac with a remote toolchain (SSH) to this box —
# CMake presets work unchanged; build/run/profile happen on the 4090.

The profiler shelf — used in every module, not just Module 6

A defining habit of this course: no lab is “done” from its screenshot alone — each one ends inside a tool. These are introduced one at a time and then assumed:

Tool	What it measures	Platform
Tracy	CPU frame timeline, zones, locks, GPU zones — the engine’s heartbeat	Mac + Linux
Google Benchmark	Microbenchmarks of CPU-side engine code (math, culling, sorting)	Mac + Linux
Xcode GPU capture	Full Metal frame debugging: encoder timeline, shader profiler, GPU counters	Mac
Instruments — Metal System Trace	CPU↔︎GPU timeline, display pacing, thermal state over time	Mac
RenderDoc	Vulkan frame debugging: event browser, resource inspection, pixel history	Linux
Nsight Graphics	GPU trace and frame debugging on the 4090, beside RenderDoc	Linux
`nvidia-smi` / `powermetrics`	Power draw and thermals — the efficiency axis	Linux / Mac
In-engine HUD	Timestamp queries (Vulkan) / counter sample buffers (Metal), rolling frame stats	Both

Recommended-reading key

Abbrev.	Book	Role
MbT	Begbie & Horga (Kodeco), Metal by Tutorials, 5th ed.	The Metal track: Swift, MTKView, MSL, and Apple-GPU idioms, tutorial-paced
C&S	Castorina & Sassone, Mastering Graphics Programming with Vulkan	The engine-architecture track: bindless resources, render graphs, GPU-driven rendering
Halladay	Halladay, Practical Shader Development	Shader intuition: vertex/fragment thinking, lighting, and effects, API-agnostic
Lengyel	Lengyel, Mathematics for 3D Game Programming and Computer Graphics, 3rd ed.	The graphics-math reference: transforms, frustums, planes, projections — read for application, the theory being Course 1’s
D&P	Dunn & Parberry, 3D Math Primer for Graphics and Game Development, 2nd ed.	The 3D math itself — coordinate spaces, transforms, quaternions, primitives. Worked as the Dunn & Parberry exercise set

All five are cited inline where used, like PEI in Course 3 — tutorial and reference books without formal exercise sets, so none is a Books-page entry; this course’s labs are the exercises. The mathematics these books apply is developed, with proofs, in Course 1 Part IX (Lessons 63–70), which every lab below points into. Chapter references are given by chapter title rather than number where editions differ — confirm against the copy in hand.

Free primary sources, cited throughout: the Vulkan Tutorial and vkguide.dev (the Vulkan on-ramp and the engine-shaped second pass), the Vulkan specification’s synchronization chapter, Apple’s Metal documentation and WWDC session videos (TBDR, GPU counters, metal-cpp), and the GLM manual.

Why two graphics APIs (and where each language is used)

Writing the same renderer against both APIs is the fastest way to learn what is essential (the pipeline, resource states, synchronization) versus what is an API’s opinion (render passes, descriptor sets vs. argument buffers, explicit vs. tracked hazards). The differences are worked, not footnoted — the coordinate conventions (Vulkan’s clip space vs. Metal’s NDC), the memory models, and the hazard-tracking philosophies each get measured head-to-head.

Vulkan labs are C++20 — the explicit API in the systems language, the combination the cross-platform engine world runs on.
Metal API-learning labs (Modules 2–3) are Swift — matching MbT 5th ed. and Xcode’s tooling, so the learning loop (edit MSL → run → GPU capture) has zero friction.
From Module 4 the engine core is C++20 with a metal-cpp backend — one scene graph, one render graph, two API backends. Swift remains the app shell; C++ owns the frame.

A note on scope: compute shaders inside the graphics APIs are used freely (culling, sorting, post-processing). CUDA lives in Course 2’s C++ repository, not here — it is the deepest tooling ecosystem for learning GPU microarchitecture, and that mental model transfers to reading Metal and Vulkan GPU captures intelligently.

Module & lab map

The course is 7 modules, 28 labs (the former Module 1, CUDA and GPU-compute foundations, moved to Course 2’s 100 exercises to learn modern C++ CUDA track; module numbers 2–7 keep their historical values). Each lab is its own page; work them in order — the engine grows monotonically from Lab 0.1’s empty CMake project to the Module 7 capstone.

Module 0 · Toolchain: C++20, CMake, and three GPU bring-ups

The project skeleton every later lab builds in, and first pixels from both graphics APIs.

Module 2 · The raster pipeline: triangles to textured meshes

The graphics pipeline proper, every lab built twice: Vulkan/C++ and Metal/Swift.

Module 3 · Shading: light, materials & post-processing

The fragment shader earns its keep: local illumination through physically based shading to the HDR post stack.

Module 4 · Engine architecture in C++20

From “a program that draws” to “an engine”: resource lifetimes, a render graph, threads, and a scene. metal-cpp arrives here.

Module 5 · Advanced real-time techniques

The techniques a modern renderer is made of — each one predicted, built on both APIs where the hardware allows, and profiled.

Module 6 · Performance engineering & profiling

The tools become the subject: find the bottleneck, name it, fix it, prove the fix.

Module 7 · Capstone: a small rendering engine, shipped

Everything converges: a streaming 3D world viewer on two APIs and two GPUs, with a staff-level performance report.

Repository structure

All lab work lives beside the Course 3 workspace in the companion repo diivanand/diiv_website_custom_courses, under a top-level course4/ folder — a self-contained CMake project whose build scaffolding is fully set up in advance (presets, pinned dependencies, per-lab stub targets that already compile), so lab time goes to the code, never the build:

diiv_website_custom_courses/course4/
  README.md                   # build quickstart
  CMakeLists.txt              # top-level project + CMakePresets.json (debug / release / profile / linux-*)
  cmake/                      # course4_options (warnings, C++20, sanitizers) + pinned FetchContent deps
  engine/                     # the C++20 engine that grows across Modules 2-7
    core/  vulkan/  metal/    #   metal/ = metal-cpp backend (Module 4+; needs third_party/metal-cpp)
  shaders/                    # glsl/ (→ SPIR-V via the compile_shaders target) and msl/
  metal-swift/                # SwiftPM package: the Swift/MTKView apps of Modules 0, 2-3 (open in Xcode to capture)
  labs/                       # one folder per lab — everything that lab produces
    lab-<M>-<N>/
      notes.md                #   predicted-vs-measured tables, reconciliation
      captures/               #   Tracy traces, .gputrace, RenderDoc .rdc, Nsight reports
      benchmarks/             #   Google Benchmark JSON, HUD stat dumps
      src/                    #   (C++ labs) the lab's executable — a stub target until you write it
  assets/                     # meshes, textures, environment maps, splat scenes (gitignored)
  docs/                       # setup notes: Vulkan SDK, metal-cpp, the Linux/4090 box, asset sources

Naming convention: everything a lab produces lives in its folder. A lab is “done” when its notes.md has every Measured cell filled and its captures/ folder holds the profiler evidence.

What you’ll be able to do at the end

Stand up a modern C++20/CMake codebase with profiling, benchmarks, and sanitizers wired in from the first commit.
Explain and use the GPU’s execution and memory model — warps, occupancy, coalescing, shared memory — and write CUDA kernels in C++ and Python that saturate a memory bus on purpose.
Drive the full graphics pipeline on both Vulkan and Metal: pipelines and pipeline state, buffers and staging, textures and samplers, depth, blending — and explain every difference between the two APIs’ models.
Implement the standard shading stack — Blinn–Phong through physically based shading with image-based lighting — and an HDR post-processing chain, in both GLSL and MSL.
Architect an engine: RAII resource lifetimes, frames in flight, a render graph with explicit barriers, multithreaded command recording, and a culled, instanced scene — one C++ core, two API backends.
Build the techniques modern renderers are made of: cascaded shadow maps, deferred shading (and its TBDR-aware variant on Apple GPUs), chunked-LOD terrain with streaming, GPU-driven culling with indirect draws, and a Gaussian-splat point renderer.
Profile like it’s part of the job — because it is: Tracy, Xcode GPU capture and Metal counters, RenderDoc, Nsight Systems/Compute — and run an optimization ladder with before/after evidence, including power.
Ship a capstone: a streaming 3D world viewer running on two APIs and two GPUs, with a written performance report a staff engineer would sign.

Relationship to Courses 1 & 2

Course 1 is the theory this course spends; Course 2 is the language discipline it builds on.

The rendering mathematics itself (Modules 2–5) is Course 1 Part IX, proved there and applied here: homogeneous coordinates, cameras, and perspective (Lesson 63), rotations and quaternions (Lesson 64), barycentrics, perspective-correct interpolation, frusta, and shadow-map geometry (Lesson 65), radiometry and the BRDFs (Lesson 66), Monte Carlo shading, the split sum, spherical harmonics, and mipmaps (Lesson 67), color and tone mapping (Lesson 68), Gaussian splatting (Lesson 69), and two-dimensional sampling — the stretch theorem, lattice sampling, and when a texture lookup aliases (Lesson 70).
Transforms, projections, and lighting math (Modules 2–3) rest on Course 1 Parts I–II — every MVP chain is a composition of linear maps; every lighting equation is inner products and orthogonal decompositions.
Depth precision, accumulation error, and fast-math tradeoffs (Modules 2, 5, 6) apply Lesson 42 — floating point and conditioning, now with a Z-buffer attached.
Texture filtering, mipmapping, and screen-space aliasing (Labs 2.4, 3.4) apply Lesson 29 — minification is resampling; the mip chain is an anti-aliasing filter bank.
Convolution post-effects and color pipelines (Labs 3.4, 5.2) apply Course 3 Lab 9.4 theory — separable Gaussian blur, sRGB and gamma, the image as a 2-D signal.
The GPU as an architecture is learned in Course 2’s CUDA track — the CPU machine model re-learned on a machine that hides latency with parallelism instead of speculation — and spent here every time a capture is read.
C++20 engine code (Module 0 onward) stands on the modern-C discipline of Course 2’s embedded-C exercises — the same care about layout, aliasing, and undefined behavior, now with RAII and templates in the toolbox.