Lab 4.1 — Resource Lifetimes & the Frame Loop
← Course 4 syllabus · Module 4 · Prev: « Lab 3.4 · Next: Lab 4.2 »
Goal
Build the engine’s spine. Modules 2–3 produced a stack of per-lab programs — each one owning its own instance, its own swapchain, its own ad-hoc cleanup order — plus a family of Swift apps on the Metal side. This lab collapses them into one C++20 engine core with two API backends: Lab 0.2’s unique_handle and generational-pool patterns now hold real GPU objects, the frames-in-flight invariant is stated precisely and enforced by construction (a deletion queue per frame slot), VMA becomes the allocator of record with live budget queries, and a per-frame descriptor allocator replaces the hand-managed descriptor sets of Module 2.
The second half is the module’s thesis made real: a metal-cpp backend. There is no MTKView in metal-cpp — the backend owns a CAMetalLayer handed over by a thin Swift/AppKit shell, and the same C++ frame loop that drives Vulkan drives Metal. The acceptance test is unglamorous on purpose: the Lab 3.1 Blinn–Phong scene, rendered by the engine, through both backends, to matching screenshots. One core, two backends — from here to the capstone.
Recommended reading
- C&S — the resources-management chapter (title-level reference; confirm against the copy in hand): pooled GPU resources, handle-based lookup, and deferred destruction in a shipping-shaped engine — this lab is that chapter with Lab 0.2’s types underneath.
- vkguide.dev — the engine-architecture chapters: deletion queues, frames in flight, and the descriptor-allocator design this lab’s version is measured against.
- VMA documentation (bundled with the SDK / GitHub) — the quick-start and recommended usage patterns pages, plus the memory-budget section (
VK_EXT_memory_budget). - Apple’s metal-cpp distribution page (developer.apple.com/metal/cpp) — the download, the single-translation-unit
*_PRIVATE_IMPLEMENTATIONrule, and the ownership conventions (NS::SharedPtr, retain/release) that must be reconciled with the engine’s RAII style. - MbT — the later engine-architecture chapters (title-level) for how the Swift-side idioms you already know map onto what the C++ backend now does by hand.
- Lab 0.2 — reread your own
unique_handle/pool notes; this lab is their payoff.
Prerequisites
Project & environment setup
- New engine component
engine/render/(targetengine_render) — the backend-agnostic layer;engine/vulkan/(targetengine_vulkan) is refactored into a backend behind it; NEWengine/metal/(targetengine_metal, macOS only) is the metal-cpp backend. - One lab executable:
labs/lab-4-1/→engine_viewer, taking--backend vulkan|metal(Metal choice macOS-only, guarded in CMake). - metal-cpp is vendored from Apple’s distribution zip (it is not a package registry artifact):
add_library(metal_cpp INTERFACE)
target_include_directories(metal_cpp INTERFACE third_party/metal-cpp)
target_link_libraries(engine_metal PRIVATE metal_cpp
"-framework Metal" "-framework QuartzCore" "-framework Foundation")Exactly one .cpp in engine_metal defines NS_PRIVATE_IMPLEMENTATION, CA_PRIVATE_IMPLEMENTATION, and MTL_PRIVATE_IMPLEMENTATION before including the headers — a link-error classic worth hitting once on purpose. - The Swift shell MetalShell (in metal-swift/, an Xcode project) is deliberately thin: window, CAMetalLayer, input forwarding, and calls across a tiny C bridge:
// engine/metal/bridge.h — the entire Swift ↔ C++ surface
void engine_attach_layer(void *ca_metal_layer); // layer pointer, bridged opaque
void engine_frame(double dt);
void engine_shutdown(void);Where results go:
| Artifact | Path |
|---|---|
| Notes, invariant statement, use-after-free postmortem, parity notes | labs/lab-4-1/notes.md |
| Screenshots (both backends), Tracy traces, GPU captures | labs/lab-4-1/captures/ |
| VMA budget dumps, frame-loop timing runs | labs/lab-4-1/benchmarks/ |
Background
The frames-in-flight invariant, precisely. With \(F\) frames in flight, frame \(N\) records into slot \(s = N \bmod F\), and before reusing slot \(s\) the CPU waits on that slot’s fence. The lifetime rule follows: a GPU resource may be destroyed only after the last fence that could possibly guard work referencing it has signaled. Operationally — a resource retired during frame \(N\) goes into slot \(N \bmod F\)’s deletion queue, and that queue is flushed at the top of frame \(N + F\), immediately after the slot’s fence wait returns. Destruction is never a vkDestroy* call at the point of “I’m done with this”; it is an enqueue. vkDeviceWaitIdle remains legal exactly twice: shutdown and swapchain recreation.
VMA as the allocator of record. Vulkan’s maxMemoryAllocationCount and per-heap granularity make one-vkAllocateMemory-per-resource a dead end; VMA suballocates from large blocks, tracks usage per heap, and — with VK_EXT_memory_budget enabled — reports what the OS will actually tolerate, not just what you asked for. Every buffer and image allocation in the engine now goes through it; the HUD grows a mem: used/budget line read from vmaGetHeapBudgets.
Descriptor management. Module 2 allocated descriptor sets by hand and never freed them. The engine’s answer is a per-frame descriptor allocator: each frame slot owns a chain of VkDescriptorPools, sets are allocated linearly during recording, and the whole chain is reset (vkResetDescriptorPool) when the slot’s fence wait returns — no individual frees, no fragmentation, and the same “keyed to the fence” logic as the deletion queue. Descriptor indexing exists and MoltenVK exposes a usable subset, but full bindless is deliberately deferred to Lab 5.4; here the discipline is per-frame and boring.
The backend seam. The interface extracted this lab is small and shaped by what both APIs can promise — described here as a shape, not code:
| Abstraction | Vulkan backend | Metal backend |
|---|---|---|
| Device | instance + physical + logical device, queues, VMA | MTL::Device, MTL::CommandQueue |
| Surface | VkSurfaceKHR + swapchain, image views |
CA::MetalLayer (handed over by the shell), nextDrawable() |
| Frame | slot fence, sync trio, command buffer, descriptor allocator, deletion queue | slot semaphore/counter, command buffer, deletion queue |
| Buffers/Images | VMA allocations behind Lab 0.2 pool handles | MTL::Buffer/MTL::Texture behind the same handles |
What Metal changes — and doesn’t. Unified memory means most buffers are storageModeShared — no staging ring, which the parity notes must call out. Automatic hazard tracking means the Metal backend needs no barriers for correctness (the explicit alternative — untracked resources with MTLFence/MTLEvent — is Lab 4.2’s subject). But the lifetime problem is identical: a MTL::Buffer released while a command buffer in flight references it is the same use-after-free, so the deletion queue is backend-agnostic and lives in engine/render/. metal-cpp’s retain/release conventions (who owns what NS::Object*, when NS::SharedPtr applies, autorelease pools around the frame) get one careful page in your notes — this is where metal-cpp bites people.
Tasks
- Extract the backend interface. Define the Device/Surface/Frame/Resource seam sketched above in
engine/render/and move the Module 0–3 Vulkan code behind it. The Lab 3.1 scene must render through the engine before any Metal work starts — refactor first, port second, never both at once. - Frame loop + deletion queues. Implement the slot-based frame loop with one deletion queue per slot, flushed after the fence wait. Route every destruction in the engine through it — grep for naked
vkDestroy/vmaDestroycalls outside the flush; the count on the frame path must be zero. - Break it on purpose. In a scratch branch, destroy a uniform buffer immediately after recording frame \(N\) instead of enqueueing it. Run with synchronization validation on. Record the exact validation message in
notes.md, then the fix, then the one-paragraph story of which fence would have made it safe and when. - VMA + budget HUD. Move all allocations to VMA (
VK_EXT_memory_budgetenabled), tag allocations by category (mesh / texture / uniform / transient), and add the HUD line showing per-heapused/budget. DumpvmaGetHeapBudgetsand the JSON stats snapshot tobenchmarks/for the reconciliation. - Per-frame descriptor allocator. Implement the pool-chain allocator, reset on slot reuse; port the Lab 3.1 descriptor usage onto it. Validation must stay silent across 10k+ frames (pool exhaustion → grow-the-chain is the case to test).
- The Swift shell + layer handover. Build
MetalShell: window,CAMetalLayersized in pixels (the RetinacontentsScaledistinction from Lab 0.4 applies), pointer across the bridge, resize and shutdown forwarded. Document the ownership rule you chose for the layer on both sides of the bridge. - metal-cpp backend. Implement the Metal side of the seam — device, queue, drawable acquisition, per-slot completion tracking, the same deletion queue — and bring the Lab 3.1 scene up through it (MSL shaders from Module 3 reused as-is).
- Parity. Same scene, same camera, both backends: capture screenshots and diff. Pin down every visible difference to a named cause (sRGB handling, clip-space vs NDC conventions from Module 2, dither) in
notes.md.
Deliverable & expected results
engine_viewer --backend vulkanand--backend metalboth render the Lab 3.1 scene; validation clean; HUD shows the budget line; screenshots + diff incaptures/.notes.mdstates the frames-in-flight invariant in your own words, carries the use-after-free postmortem, and the parity accounting.
| Quantity | Predicted | Measured |
|---|---|---|
| Engine frame-loop CPU cost vs. Module 2’s hand-rolled loop (Tracy, same scene) | ≈ equal — the abstraction is interfaces and pools, not work; any regression is a bug to find, not a tax to accept | … |
| VMA-reported device-memory usage vs. hand-computed asset size (mesh bytes + texture bytes with mips, from the asset files) | within ~10–20% above hand-computed — block granularity, alignment, depth/swapchain images account for the gap, itemized | … |
| Deliberate use-after-free under sync validation | caught before the fix with a named hazard message; silent after | … |
| metal-cpp backend vs. Module 3 Swift app, frame time, same scene | ≈ equal — same GPU work, same API underneath; metal-cpp is bindings, not a layer | … |
Profiling & performance
Tracy zones now belong to the engine, not the lab: wait_fence, flush_deletions, acquire, record, submit, present — named identically in both backends so traces are comparable side by side. Take one capture per backend on the Lab 3.1 scene and archive both; every Module 4–7 lab reads against these. Add a Tracy plot for the VMA used-bytes number — watching it step when assets load and not creep frame-over-frame is the leak test.
Analysis & reconciliation
Reconcile the VMA number against your hand-computed asset total, itemizing the overhead: alignment, block slack, the images the engine owns that the asset list doesn’t (depth, swapchain). Explain the frame-loop comparison honestly — if the engine loop costs more than the hand-rolled one, find the zone that grew. For Metal-vs-Swift, confirm from the two GPU captures that encoder counts and pass structure match, so any CPU-side delta is bridge overhead, and say whether it’s measurable at all. Close with the invariant restated: for each resource class, which fence guards it, and where in the code that guarantee lives.
Going further
- Run
engine_viewer --backend vulkanunchanged on the Linux desktop (RTX 4090) — the backend seam’s first portability dividend; diff the VMA heap layout (a discrete device-local VRAM heap plus host-visible heaps) against the Mac’s single unified heap. - Pressure the budget: allocate throwaway textures until
vmaGetHeapBudgetsusage crosses budget, and record what actually happens on each platform (MoltenVK vs. native) — eviction, failure, or slowdown. - Make \(F\) a runtime knob: run \(F = 2\) vs. \(3\) and measure latency (input-to-photon proxy) against memory and CPU wait time — the tradeoff Lab 6.1 formalizes.