Lab 2.3 — Transforms, Depth & the Camera

← Course 4 syllabus · Module 2 · Prev: « Lab 2.2 · Next: Lab 2.4 »

Goal

Put the mathematics to work: the model–view–projection chain as a composition of linear maps in homogeneous coordinates (Course 1 §1 is assumed mastered — this lab applies it, it does not re-derive it), delivered to the GPU through uniform buffers with a correct per-frame update strategy, a depth buffer on both APIs so a grid of cubes overlaps honestly, and a camera you can actually fly. And this is the coordinate-system lab: Vulkan’s clip space is Y-down with 0–1 depth, Metal’s NDC is Y-up with 0–1 depth, and GLM was born speaking OpenGL — every renderer that targets both APIs must pick one deliberate place where the conventions are reconciled, and this lab is where you pick yours. The deliverable is unusually strict: the same scene, both APIs, and the screenshots must match pixel-for-pixel in orientation — no “close enough”.

Recommended reading

• Lengyel — the transforms chapters (linear/affine transforms, quaternions) and the projections/frustum material (title-level; 3rd-ed. — confirm against the copy in hand). Read for application: the theory is Course 1’s, but Lengyel’s treatment of projection matrices and rotation representations is the working reference.
• D&P — the coordinate-space and orientation chapters: the gentlest and most careful telling of handedness, spaces, and Euler-vs-quaternion tradeoffs anywhere in print.
• Vulkan Tutorial — the “Uniform buffers” and “Depth buffering” chapters; note where the tutorial’s GLM usage silently assumes OpenGL conventions — finding those spots is practically an exercise in itself.
• MbT — the coordinate-spaces/transforms and depth chapters (title-level). The GLM manual’s sections on GLM_FORCE_DEPTH_ZERO_TO_ONE and the projection functions’ RH/LH/ZO/NO variants — five minutes that prevent the classic upside-down-and-half-clipped scene.

Prerequisites

• Lab 2.2 — buffers, VMA, push constants on both APIs.
• The Lab 2.1 postmortems fresh in mind — the negative-viewport experiment and the Y-flip observation become policy here.

Project & environment setup

• Vulkan: labs/lab-2-3/ → vk_cubes; engine/vulkan/ gains descriptor-set machinery (layout, pool, per-frame sets) and a depth-image helper; engine/core/ gains the camera controller and a math/ corner where the convention policy lives.
• Metal: metal-swift/MetalCubes/ — MTKView gets depthStencilPixelFormat = .depth32Float; a MTLDepthStencilState enters the encoder setup.
• GLM policy, set project-wide in CMake now: GLM_FORCE_DEPTH_ZERO_TO_ONE (both APIs are 0–1 depth; GLM’s default −1…1 must die in this codebase) and GLM_FORCE_RADIANS. Record both in the README’s build table.

Where results go:

Artifact	Path
Notes, convention policy write-up, depth-precision table	labs/lab-2-3/notes.md
The pixel-match screenshot pair, captures	labs/lab-2-3/captures/

Background

The chain. Model space → world (per-object \(M\)) → view (camera \(V\)) → clip (\(P\)), one composed matrix per object per frame: \(\text{MVP} = P\,V\,M\). Everything here is Course 1 §1 — composition of linear maps, with homogeneous coordinates buying translation and projection admission into matrix algebra.

The projection matrix, right-handed view space (camera looking down \(-z\)), 0–1 depth, \(f = \cot(\theta_y/2)\), aspect \(a\):

\[ P \;=\; \begin{bmatrix} f/a & 0 & 0 & 0\\[2pt] 0 & f & 0 & 0\\[2pt] 0 & 0 & \dfrac{z_f}{z_n - z_f} & \dfrac{z_n z_f}{z_n - z_f}\\[6pt] 0 & 0 & -1 & 0 \end{bmatrix} \]

Verify by hand (do it — it’s two substitutions) that view-space \(z=-z_n\) lands at depth \(0\) and \(z=-z_f\) at depth \(1\) after the divide by \(w = -z\). This is glm::perspective under GLM_FORCE_DEPTH_ZERO_TO_ONE.

Depth is hyperbolic in distance. After the perspective divide, stored depth as a function of view distance \(z\) is

\[ d(z) \;=\; \frac{z_f}{z_f - z_n}\left(1 - \frac{z_n}{z}\right), \]

a \(1/z\) curve: for \(z_f \gg z_n\), the slab from \(z_n\) to \(2z_n\) already consumes about half the entire depth range, leaving the far 99% of the scene to share the rest. Combine that with floating-point spacing (Course 1 §3 — precision is also densest near zero) and you get the classic pathology: near-field over-precision, far-field z-fighting, and a hard rule — never set \(z_n\) smaller than you need. Reverse-Z, which turns the float distribution from enemy to ally, is deferred to Going further and formalized later in the course.

The Y question. Vulkan clip space is Y-down; Metal NDC is Y-up; GLM builds Y-up (OpenGL) matrices. Two standard Vulkan-side fixes: negate the projection’s \([1][1]\) entry, or use a negative viewport height (Lab 2.1’s experiment, VK_KHR_maintenance1, core 1.1) — each flips winding side-effects differently, so cull mode must be set with the choice, not after it. The engine rule this course adopts: conventions are fixed in one named place (the projection helper in engine/core/math/), every shader and API backend consumes the result, and nobody flips anything anywhere else. Which fix you choose matters less than that there is exactly one.

Uniform buffers and frames in flight. The per-frame data (at minimum \(V\), \(P\), camera position) outgrows push constants as scenes grow. But Lab 0.3’s loop keeps two frames in flight — writing one UBO while the GPU reads it is a race. The standard fix: a ring of one UBO slice per in-flight frame (or one buffer, per-frame offsets aligned to minUniformBufferOffsetAlignment — query it; 256 B is a common answer), indexed by the frame counter. Metal has the same race and the same fix — shared storage does not mean synchronized — with a ring of buffers or offsets guarded by the completed-handler/semaphore pattern from Lab 0.4’s loop.

Tasks

Vulkan (C++20)

1. The convention policy. Write the projection/view helpers in engine/core/math/ with the Y-flip strategy chosen, documented in a comment block and in notes.md: which fix, what it does to winding, what cull mode is therefore correct. This file is the only place the word “flip” is allowed to appear.
2. UBO ring. Descriptor-set layout for a per-frame UBO; a ring of MAX_FRAMES_IN_FLIGHT slices; write via persistently mapped VMA allocation; bind the right slice per frame. Prove the race exists first: use a single UBO, animate the camera fast, and describe the artifact (or validation/sync-validation complaint) you get — then fix it with the ring.
3. Depth. Create the depth image (D32_SFLOAT, device-local) and attach it via dynamic rendering; enable depth test and write in the pipeline; handle resize (depth image recreates with the swapchain).
4. The scene. A cube mesh (indexed, from Lab 2.2’s machinery) instanced as an \(8 \times 8\) grid of cubes via per-object model matrices (push constants are fine for \(M\) at this count); rotate them at distinct phases so overlap is everywhere. Without depth test: a shuffled mess — screenshot it. With: correct — screenshot that too.
5. Camera. An orbit camera (yaw/pitch about a target — Euler angles are fine here, and say why: no roll, gimbal lock unreachable by construction) and a fly camera (WASD + mouse-look). Store orientation your chosen way, but write the quaternion-vs-Euler paragraph in notes.md from the Lengyel/D&P reading: what breaks first with Euler (composed roll, interpolation), what quaternions buy (slerp, no gimbal lock, renormalization instead of drift), and why Module 5’s shadow-camera work will thank you.

Metal (Swift)

6. Same scene, same math. Reuse the same matrix conventions — the cleanest proof of the one-place policy is that the Metal path needs no flip beyond what the policy already decided. Per-frame uniforms via the ring-of-buffers pattern; depth via the view’s depth format plus a MTLDepthStencilState (less compare, write on).
7. Prove your handedness. Compose the acid test: an asymmetric scene (cubes colored by grid coordinate, one corner marked, camera at a known offset — not on an axis of symmetry). Screenshot both APIs from the identical camera pose. The pair must match pixel-for-pixel in orientation: same corner up-left, same faces toward camera, same rotation direction. Any mirror-image or upside-down discrepancy is a convention bug — find it in your policy file, not with a local patch. Both screenshots plus a one-paragraph “why they match” go in captures/ and notes.md.

Deliverable & expected results

• vk_cubes and MetalCubes flying through the same cube grid, depth-correct, with the matched screenshot pair as evidence; the convention write-up, race description, and quaternion-vs-Euler paragraph in notes.md.

Quantity	Predicted	Measured
Depth value at \(z = 2z_n\) (from \(d(z)\), \(z_f \gg z_n\))	\(\approx 0.5\) — half the range in the first doubling	… (evaluate your \(d(z)\) numerically)
Depth range covering the far half of the scene	a thin sliver near \(1.0\) — read it off \(d(z)\)	…
UBO ring update cost per frame (Tracy zone)	microseconds — a small memcpy; if it isn’t, something’s wrong	…
Single-UBO race symptom	tearing/jumping transforms or a sync-validation report	…
Screenshot orientation match	pixel-for-pixel	…

Profiling & performance

Capture the vertex stage this lab: in Xcode GPU capture, find the vertex function’s cost and the geometry statistics for the cube grid (vertices in, primitives out); on Vulkan, RenderDoc’s mesh viewer on the Linux desktop (RTX 4090) — or the Tracy GPU zone on the Mac — for the same. The number to internalize: at a few thousand vertices, vertex work is nothing — the frame is still swapchain-bound. Record the depth attachment’s store action on the Metal side (does the depth buffer need .store? It doesn’t — .dontCare it and note why, tying back to Lab 0.4’s TBDR observation).

Analysis & reconciliation

Reconcile the measured depth values against the \(d(z)\) curve — compute \(d\) for three distances by hand, then read the same fragments’ depth in a capture (RenderDoc pixel history or Xcode’s depth attachment view). Explain the near-plane rule in one sentence a junior engineer would remember. Reconcile the UBO race: draw the two-frames-in-flight timeline (Lab 0.3’s diagram, now with a buffer write in it) showing exactly which write raced which read, and how the ring breaks the dependency. Close with the convention policy reviewed against both APIs’ documentation — every flip accounted for, none duplicated.

Going further

• Reverse-Z: swap to a far-at-0 mapping (greater compare, cleared to 0) and re-measure the far-field depth distribution — the fix for everything the \(1/z\) argument diagnosed; formalized later in the course alongside D32_SFLOAT vs. D24 tradeoffs.
• Implement slerp between two camera poses (quaternion path only) and eyeball it against lerp-plus-normalize — Lengyel’s interpolation discussion, made visible.
• Add a debug HUD line printing the camera pose, and a hotkey dumping it — reproducible captures need reproducible cameras, a habit Module 6 depends on.