Lab 3.2 — Normal Mapping & Tangent Space

← Course 4 syllabus · Module 3 · Prev: « Lab 3.1 · Next: Lab 3.3 »

Goal

Give flat triangles the shading of detailed surfaces: tangent-space normal mapping, where a texture stores per-texel normals in a coordinate frame attached to the surface, and the shader rebuilds that frame — the TBN basis — to carry the stored normal into world space. The mathematical content is pure Course 1 §1 and this lab says so up front: TBN is nothing but a change of basis, and every classic normal-mapping bug is a change-of-basis error wearing a costume.

The engineering content is the conventions — where tangents come from, what the green channel means, which handedness the bitangent has — and, most valuably, the debugging repertoire: visualizing the frame as colors, keeping a known-good test texture, and deliberately producing the classic failure gallery so each bug’s visual signature is learned before it appears by accident. The habits built here (regression-test against a flat map, visualize every intermediate vector) carry through every shading lab that follows.

Prerequisites

  • Lab 3.1 done on both tracks — lighting that responds to normals is the instrument that makes normal maps visible at all; the linear-space/sRGB discipline is assumed (and matters here: normal maps are data, not color — they must not be sRGB-decoded).
  • Lab 2.4’s texture/sampler machinery, and Lab 2.5’s glTF loading — tangents arrive through the same pipeline.

Project & environment setup

  • Extend the vertex layout with a vec4 tangent attribute; extend the glTF loader to read TANGENT when present. For meshes without tangents, either generate them (MikkTSpace’s reference implementation is a small C library — fair game as a dependency via FetchContent) or restrict this lab’s assets to tangent-carrying glTF files; record which route you took in notes.md.
  • Assets: one normal-mapped glTF model (brick/stone/panel surfaces show the effect best) and one known-good test normal map — a flat map (the constant \((0.5, 0.5, 1)\) texel) plus a simple bump pattern whose expected lighting you can reason about by hand. Store sources and provenance in docs/.
  • Texture-format discipline: albedo stays sRGB; the normal map binds as UNORM, not sRGB — on both APIs. This is the lab’s most common silent bug; make it a checklist item and check it in the frame debugger, not the source code.

Where results go:

Artifact Path
Notes, TBN derivation, failure-gallery postmortems labs/lab-3-2/notes.md
A/B screenshots, failure gallery, debug-view captures labs/lab-3-2/captures/

Background

The frame

At each surface point, take the unit normal \(\mathbf{n}\), a unit tangent \(\mathbf{t}\) chosen to point along increasing texture coordinate \(u\), and the bitangent \(\mathbf{b}\) completing the frame. A tangent-space normal \(\mathbf{n}_{ts}\) read from the texture maps to world space by the linear map whose columns are the frame vectors:

\[ \mathbf{n}_{w} \;=\; \begin{bmatrix} \mathbf{t} & \mathbf{b} & \mathbf{n} \end{bmatrix} \mathbf{n}_{ts} \;=\; n_{ts,x}\,\mathbf{t} + n_{ts,y}\,\mathbf{b} + n_{ts,z}\,\mathbf{n}. \]

That is the entire theory — a change of basis from the surface’s local frame to world coordinates, §1 verbatim. When the frame is orthonormal the matrix is orthogonal, so its inverse is its transpose:

\[ \mathbf{n}_{ts\text{-space}} = \begin{bmatrix} \mathbf{t} & \mathbf{b} & \mathbf{n} \end{bmatrix}^{\mathsf{T}} \mathbf{x}_{w}, \]

which is how lighting vectors can instead be pulled into tangent space in the vertex shader — the cheaper direction when many texels share one frame. Evaluating that tradeoff is one of the tasks.

Where the tangent comes from

The tangent is defined by the texture parameterization: it is the direction in which \(u\) increases across the surface. Per triangle, write the edge vectors in terms of the UV deltas —

\[ \Delta\mathbf{p}_1 = \Delta u_1\,\mathbf{t} + \Delta v_1\,\mathbf{b}, \qquad \Delta\mathbf{p}_2 = \Delta u_2\,\mathbf{t} + \Delta v_2\,\mathbf{b}, \]

a \(2{\times}2\) linear system whose solution is the triangle’s \(\mathbf{t}\) and \(\mathbf{b}\) (Lengyel derives it cleanly). Because neighboring triangles disagree slightly, per-vertex tangents are averaged and then re-orthonormalized against \(\mathbf{n}\) with one Gram–Schmidt step:

\[ \mathbf{t}' \;=\; \operatorname{normalize}\bigl(\mathbf{t} - (\mathbf{t}\cdot\mathbf{n})\,\mathbf{n}\bigr). \]

Different tools make different choices in that averaging — which is exactly why MikkTSpace exists: a fixed, documented algorithm so the baker’s frame and the renderer’s frame agree. “Same mesh, different tangent generator” is a real bug category, not a hypothetical.

What the texture stores

Each texel holds a unit vector with components in \([-1,1]\), stored as UNORM \([0,1]\); the shader decodes

\[ \mathbf{n}_{ts} \;=\; 2\,\mathbf{c} - \mathbf{1}. \]

A flat surface encodes \((0,0,1)\) — the familiar lavender texel \((0.5, 0.5, 1)\). Two convention traps to internalize now:

  • The green channel. OpenGL-style (\(+Y\) up in the tangent frame) vs. DirectX-style (\(-Y\)) — glTF specifies the former; textures sourced from other ecosystems are frequently the latter, and the flip is invisible in the texture viewer.
  • Bitangent handedness. glTF’s TANGENT.w is \(\pm 1\) and the bitangent is \(\mathbf{b} = w\,(\mathbf{n} \times \mathbf{t})\). Mirrored UV islands flip the sign, so hardcoding \(w = 1\) works on exactly half of real content — the worst kind of bug, the kind that passes on the test asset.

Interpolation breaks unit length

The rasterizer interpolates the frame vectors linearly across the triangle; the interpolated vectors are no longer unit length, nor exactly orthogonal. Renormalizing in the fragment shader is the standard fix; skipping it is one of the scheduled failure images — the artifact is subtle on dense meshes and glaring on coarse ones, which is itself a lesson about where such bugs hide.

Tasks

Same ladder both tracks; the two apps must land on matching images.

Vulkan (C++20)

  1. Tangents through the pipeline. TANGENT into the vertex layout and the glTF loader; verify against the flat test map first — a normal-mapped surface with the flat map must render identically to Lab 3.1. This is the regression test the rest of the lab leans on; screenshot the proof.
  2. Debug views. Toggleable visualizations: \(\mathbf{n}\), \(\mathbf{t}\), \(\mathbf{b}\) each as RGB, and the raw sampled normal-map texel. Screenshot each on the test model — tangents should flow visibly along the UV direction; discontinuities should sit on UV seams and nowhere else.
  3. TBN construction choice. Implement the frame per-vertex (interpolate, renormalize per fragment) and compare with re-orthonormalizing fully per fragment; note any visible difference and the shader-cost difference for measurement below.
  4. The A/B. Normal-mapped vs. vertex-normal renders of the same model, same lights, same camera — the canonical before/after pair, plus a grazing-light pair where the effect is strongest.
  5. The failure gallery. Deliberately, one at a time, each screenshotted and captioned in notes.md:
    • flipped green channel — bumps light as dents along one axis only;
    • skipped renormalization — subtly dimmed, blotchy specular on curved surfaces;
    • wrong handedness (ignore TANGENT.w) — correct on some UV islands, inverted on mirrored ones;
    • normal map bound as sRGB — a bias/contrast distortion of the vectors, everything slightly wrong everywhere.

Metal (Swift)

  1. Tangents in through the MSL vertex function and the Metal-side glTF path; the flat-map regression test again, screenshotted.
  2. Debug views matching the Vulkan set — the shader-debugging habit transfers wholesale, and the two APIs’ debug views should agree pixel-for-pixel.
  3. TBN choice mirrored; the comparison numbers come from Xcode’s shader profiler below.
  4. The A/B pair, visually matched to the Vulkan track’s.
  5. Failure gallery — at minimum the green-flip and handedness cases; confirm the visual signatures match the Vulkan captures (they must: nothing here is API-dependent, and confirming that is the point).

Deliverable & expected results

  • Both apps rendering the normal-mapped model correctly, with the debug-view toggles and the flat-map regression test passing.
  • captures/ holding the A/B pairs and the captioned failure gallery; notes.md holding the TBN change-of-basis derivation in your own notation and the per-failure postmortems.
Quantity Predicted Measured
Silhouette with vs. without normal map identical — normal mapping alters shading only, never geometry; check the outline pixel-for-pixel …
Flat test map vs. Lab 3.1 render identical images …
Extra texture memory for a 2048² RGBA8 normal map \(2048^2 \times 4\) B \(= 16\) MiB, \(\times \tfrac{4}{3}\) with mips \(\approx 21.3\) MiB …
Bitangent handedness on mirrored UV islands sign follows glTF TANGENT.w \(= \pm 1\); hardcoding \(+1\) inverts bumps exactly on mirrored islands …
Per-fragment vs. per-vertex TBN cost small per-fragment ALU delta; visible quality difference only under coarse tessellation …

Profiling & performance

Two measurements. First, shader cost: Xcode GPU capture’s shader profiler on the Metal fragment function, comparing the per-vertex-TBN and per-fragment-TBN variants — attribute the delta to specific lines. On the Vulkan side, the same comparison via RenderDoc on the Linux desktop (RTX 4090) (pass durations from the event browser; RenderDoc’s texture viewer is also the fastest way to confirm the normal map really bound as UNORM — inspect the format in the resource panel rather than trusting the loader).

Second, bandwidth: the normal map adds a texture fetch per fragment — find it in the capture’s texture-traffic counters and square it with the memory prediction above. Tracy zones as always; GPU timestamp queries arrive in Lab 6.3.

Analysis & reconciliation

The reconciliation here is mostly visual and the standard is precision: for each failure-gallery image, write the causal chain from convention error to visual signature in two sentences — e.g. the green flip negates \(n_{ts,y}\), which mirrors the stored normal about the \(u\) axis, so shading responds as if bumps along \(v\) were inverted, while bumps along \(u\) stay correct.

Confirm the silhouette prediction honestly (zoom in; the outline must not move) and state in one sentence why it cannot — which stage of the pipeline normal mapping does and does not touch. Reconcile the measured TBN-variant cost against your predicted ALU delta. Close with the judgment call: which construction does your engine standardize on, and why — and does the answer change between the Mac’s GPU and the 4090?

Going further

  • Two-channel normal maps: store \((x, y)\) only and reconstruct \(z = \sqrt{1 - x^2 - y^2}\) — the BC5-compression convention; derive what the reconstruction assumes about the hemisphere.
  • Run MikkTSpace generation against a mesh that ships tangents and diff the two frames as a color visualization — how much do generators disagree in practice?
  • Object-space normal maps: store world-frame-of-the-model normals directly, no TBN at all — work out what breaks (deformation, mirrored UVs, texture reuse across meshes) and why the tangent-space form won anyway.
  • Mip levels vs. normal maps: averaging unit vectors does not yield unit vectors, so mipped normal maps get smoother at distance — connect this to §8 and to Lab 3.3, where “lost” normal variance is exactly what roughness represents.
  • Parallax mapping as the next rung (Halladay and LearnOpenGL both cover it): the height field starts lying about position, not just orientation — and its silhouette limitation states exactly where the normal-mapping trick’s jurisdiction ends.
  • Self-shadowing check: what happens when \(\mathbf{n}_w \cdot \mathbf{l} > 0\) but the geometric normal faces away from the light? Produce the artifact and propose the standard mitigation.