Lab 3.3 — Physically Based Rendering & Image-Based Lighting

← Course 4 syllabus · Module 3 · Prev: « Lab 3.2 · Next: Lab 3.4 »

Goal

Replace Blinn–Phong with the shading model the industry actually ships: the metallic-roughness PBR model as glTF 2.0 specifies it — Cook–Torrance microfacet specular built from a GGX distribution, a Smith geometry term, and Schlick’s Fresnel, with energy conservation and the metal/dielectric split doing the work that ad-hoc \(k\)’s did in Lab 3.1. Then extend it from point lights to whole environments: image-based lighting, where an HDR environment map is precomputed into a diffuse irradiance map, a prefiltered specular mip chain, and a BRDF LUT — the split-sum approximation that made real-time IBL possible.

This is the module’s big lab — budget multiple sessions and say so in the log; the task list is split so the halves have their own acceptance images. It is also a milestone of a second kind: the three precompute passes are this course’s first compute shaders inside the graphics APIs — Module 1’s CUDA mental model (grids of threads, throughput thinking) arriving in Vulkan and Metal dress. The payoff image is the canonical one: a metallic × roughness sphere grid under an environment, every sphere explaining one cell of the parameter space.

Prerequisites

  • Lab 3.1 and Lab 3.2 done on both tracks — linear-space discipline, the light array, and tangent-space normals all feed directly in.
  • Lab 2.4’s mipmap machinery understood — the prefiltered chain repurposes mip levels as a roughness axis.
  • Comfort with compute dispatch from Module 1 — the API syntax is new here, the model is not.

Project & environment setup

  • Shaders: pbr.vert/pbr.frag plus compute stages equirect_to_cube.comp, irradiance.comp, prefilter.comp, brdf_lut.comp (GLSL), and the MSL counterparts in a PBR.metal / IBLPrecompute.metal pair.
  • The compute pipelines need API plumbing Module 2 never built: compute pipeline creation, storage-image/texture writes, and dispatch. That scaffolding (pipeline creation, the barriers between passes) is fair game to study from vkguide and MbT; the kernels themselves are yours.
  • Assets: 2–3 HDR equirectangular environment maps (.hdr — stb_image loads them as float; Poly Haven is the standard free source; record provenance in docs/), and a glTF model with full metallic-roughness textures. The sphere grid is generated, not loaded.
  • Material system: extend Lab 2.5’s loader to consume the glTF material textures — base color (sRGB), metallic-roughness (UNORM — the channels are data: roughness in G, metallic in B, per the spec), normal (UNORM, from 3.2), plus occlusion and emissive. The BRDF LUT may alternatively be loaded precomputed from disk — an allowed shortcut; flag it in notes.md if taken.

Where results go:

Artifact Path
Notes, BRDF derivations, precompute-pass postmortems labs/lab-3-3/notes.md
Sphere-grid screenshots, cubemap face dumps, captures labs/lab-3-3/captures/
Precompute timings labs/lab-3-3/benchmarks/

Background

The BRDF

The metallic-roughness model evaluates, per light direction,

\[ f(\mathbf{l}, \mathbf{v}) \;=\; k_d\,\frac{\mathbf{c}_{\text{albedo}}}{\pi} \;+\; \frac{D(\mathbf{h})\,F(\mathbf{v}, \mathbf{h})\,G(\mathbf{l}, \mathbf{v})}{4\,(\mathbf{n}\cdot\mathbf{l})(\mathbf{n}\cdot\mathbf{v})} . \]

D — the normal distribution. GGX/Trowbridge–Reitz, with \(\alpha = \text{roughness}^2\):

\[ D(\mathbf{h}) \;=\; \frac{\alpha^2}{\pi\bigl((\mathbf{n}\cdot\mathbf{h})^2(\alpha^2 - 1) + 1\bigr)^2} . \]

Now Lab 3.1’s foreshadowing pays off: \(\mathbf{n}\cdot\mathbf{h}\) was always the microfacet question — what fraction of microscopic mirrors are oriented to reflect \(\mathbf{l}\) into \(\mathbf{v}\) — and \(D\) is that fraction’s distribution, with Blinn’s arbitrary exponent \(p\) replaced by a roughness parameter with physical meaning and heavier tails (GGX’s long highlight falloff is most of what makes PBR look right).

F — Fresnel. Schlick’s approximation:

\[ F(\mathbf{v}, \mathbf{h}) \;=\; F_0 + (1 - F_0)\,\bigl(1 - \mathbf{v}\cdot\mathbf{h}\bigr)^5 , \]

reflectance rising to 1 at grazing angles for every material — check the sphere silhouettes for it.

G — geometry. The Smith shadowing-masking term: microfacets occlude each other, increasingly so with roughness and at grazing angles. Use the height-correlated form or the Schlick-GGX form LearnOpenGL uses — pick one, cite the choice, and note in notes.md that the difference is small at moderate roughness (the furnace test below will expose what both leave out).

Energy conservation and the metal/dielectric split. The bookkeeping Blinn–Phong never did:

\[ k_d \;=\; (1 - F)\,(1 - \text{metallic}), \]

so light reflected specularly is not also scattered diffusely. Dielectrics get a near-constant \(F_0 \approx 0.04\) and keep their albedo as diffuse color; metals take \(F_0\) from the albedo and have no diffuse term at all — one metallic parameter interpolating two different physical regimes.

The rendering equation, restricted

What any lighting does to the surface is the hemisphere integral

\[ L_o(\mathbf{v}) \;=\; \int_{\Omega} f(\mathbf{l}, \mathbf{v})\, L_i(\mathbf{l})\, (\mathbf{n}\cdot\mathbf{l})\, d\omega_l . \]

For point and directional lights the integral collapses to a sum — that is “direct-light PBR”, the first half of the tasks. IBL is the refusal to collapse it: \(L_i\) becomes an entire environment map, and the integral must be precomputed.

The split sum

Karis’s approximation factors the specular integral into two independently precomputable pieces:

\[ \int_{\Omega} f\, L_i\, (\mathbf{n}\cdot\mathbf{l})\, d\omega \;\approx\; \underbrace{\Bigl(\tfrac{1}{N}\textstyle\sum_k L_i(\mathbf{l}_k)\Bigr)}_{\text{prefiltered environment}} \;\cdot\; \underbrace{\int_{\Omega} f\,(\mathbf{n}\cdot\mathbf{l})\, d\omega}_{\text{BRDF LUT}(\,\mathbf{n}\cdot\mathbf{v},\ \text{roughness}\,)} . \]

The first factor is stored as a cubemap whose mip levels index roughness — rougher → blurrier reflection → lower mip: §8’s filter-bank idea, bent around a sphere. The second is a 2-D lookup returning a scale and bias on \(F_0\), depending only on \(\mathbf{n}\cdot\mathbf{v}\) and roughness — environment-independent, hence computable once, ever.

The diffuse side is simpler: a low-resolution irradiance cubemap, the environment cosine-convolved over the hemisphere per normal direction. Three precomputes, all embarrassingly parallel over output texels — which is exactly why they are the course’s first graphics-API compute shaders.

Tasks

Scoped in two halves: direct-light PBR (sessions 1–2), then IBL (sessions 3+). Do not start IBL until the sphere grid under point lights looks right — a broken BRDF under a correct environment is undebuggable.

Vulkan (C++20)

  1. Material system. glTF metallic-roughness textures through the loader into the material descriptor set; correct sRGB/UNORM format per texture (the 3.2 checklist item, now with more textures to get wrong). Verify formats in RenderDoc, not in the loader code.
  2. Direct-light PBR. Cook–Torrance under Lab 3.1’s light array. First acceptance image: the sphere grid — metallic 0→1 down one axis, roughness 0→1 across the other — under 2–3 point lights. Verify the two regimes: dielectric-smooth corner shows a tight neutral highlight over albedo; metal-rough corner shows a broad, albedo-tinted sheen and no diffuse.
  3. Equirect → cubemap. First compute pass: dispatch over the six faces, sampling the equirectangular HDR by direction. Dump the faces to disk and eyeball the seams before proceeding.
  4. Irradiance convolution. Second compute pass, to a small cubemap (32²–64² per face is traditional — reason about why so little resolution suffices for a cosine-blurred signal before accepting the folklore).
  5. Prefiltered chain + LUT. Dispatch per mip level with increasing roughness; then the BRDF LUT (or load it precomputed). Barriers between passes are the Vulkan lesson here — each pass reads what the previous one wrote, and synchronization validation will referee.
  6. Composite. Ambient term = irradiance × diffuse + prefiltered fetch × LUT-scaled \(F_0\); the sphere grid under each environment, with a skybox pass showing the source environment behind it. Canonical screenshot into captures/.

Metal (Swift)

  1. Material system mirrored into argument-buffer bindings; same per-texture format discipline, verified in the GPU capture’s resource inspector.
  2. Direct-light PBR in MSL; the sphere grid must match the Vulkan render — the strongest cross-API acceptance test yet, because the equations leave no artistic slack.
  3. Compute passes as MTLComputeCommandEncoder dispatches writing cube textures — note how much of Vulkan’s barrier ceremony Metal’s automatic tracking absorbs, and record the observation for Module 4’s abstraction layer.
  4. Irradiance, prefilter, LUT as above; per-mip texture views are the counterpart machinery on this side.
  5. Composite + skybox; matched canonical screenshot.

Deliverable & expected results

  • Both apps: the metallic × roughness sphere grid lit by direct lights and by each environment map, plus a textured glTF model under IBL; skybox visible.
  • notes.md: derivation notes for \(D\), \(F\), \(G\) — including the hand-worked limits: what does GGX do as roughness → 0 and → 1? what does Schlick give at normal incidence and at grazing? — plus the pass-ordering diagram with barriers, and the postmortems.
Quantity Predicted Measured
Prefiltered map mip count \(\log_2(\text{face size}) + 1\) — e.g. a 128² face chain gives 8 levels; record how many you map to the roughness axis …
Rough metal vs. smooth dielectric metal: no diffuse, albedo-tinted broad reflection; dielectric: albedo diffuse + neutral tight highlight — the grid’s opposite corners …
Fresnel at grazing angles every sphere trends mirror-like at the silhouette, regardless of metallic …
Precompute cost one-time at load (or offline); per-frame IBL cost ≈ a few texture fetches — effectively zero marginal cost vs. Lab 3.1’s frame …
Cross-API image match indistinguishable sphere grids …

Profiling & performance

Two subjects. The precomputes: wrap each compute pass in Tracy zones and record wall time per pass per environment. On the Metal side take an Xcode GPU capture of the load sequence and read each dispatch’s cost off the GPU timeline — first practice attributing compute (not raster) work in the capture tools; open the shader profiler on the prefilter kernel, the most expensive of the three. On the Vulkan side, capture the same sequence with RenderDoc on the Linux desktop (RTX 4090) and inspect the storage-image writes between passes — the barrier structure is visible in the event browser.

The steady state: confirm the per-frame claim — capture a rendered frame on both APIs and find the IBL cost hiding inside the fragment shader (the cubemap and LUT fetches). GPU timestamp queries formalize all of this in Lab 6.3; until then, captures are the record.

Analysis & reconciliation

Reconcile the mip-count prediction first, then write the deeper reconciliation this lab exists for: where did the energy go? Pick one sphere (mid-roughness dielectric) and account for its rendered color under one environment — which fraction came from irradiance × albedo, which from the prefiltered fetch — and verify by disabling each term: does the image your accounting predicts appear?

Run the classic furnace test: under a uniform pure-white environment, energy-conserving spheres should render near-uniformly white. Describe your deviation and its likely source — the \(G\)-term choice, or the single-scattering assumption both standard forms share (rough surfaces lose the light that should have bounced between microfacets; the fix is a scheduled extension below). Finally, reconcile the two APIs’ precompute timings against each other and against a naive per-texel work estimate — the first cross-GPU compute comparison outside CUDA.

Going further

  • Multiple-scattering energy compensation — the furnace test’s missing energy at high roughness has a standard published fix; research and implement it behind a toggle, and re-run the furnace.
  • KHR_materials_emissive_strength and the occlusion texture — finish the glTF material feature set.
  • Compare your BRDF LUT against a published reference image; then substitute Karis’s analytic environment-BRDF approximation and measure what changes.
  • Spherical-harmonic irradiance — 9 coefficients instead of a cubemap: §1’s orthogonal projections, on the sphere; measure memory and per-frame cost against the cubemap version.
  • Run the precompute suite on both the 4090 and the Mac and compare compute throughput — a preview of Module 6’s cross-GPU ladder.