Syllabus
C++20 + Vulkan Graphics/Physics Engine for Self-Driving Car Simulation — 10-Week Detailed Schedule
Full week-by-week plan. Each entry lists the theme and reading. Detailed theory exercises, implementation tasks, benchmarks, and definitions of done live in the weekly lecture notes.
Prerequisites — Course 5. This course assumes the mathematical foundations are already in hand from Course 5: linear maps and homogeneous transforms (the rendering and camera math), classical mechanics (vehicle physics), differential geometry of curves (road/lane geometry, curvature), graph algorithms (lane graph, routing), and geometrical optics / image formation (camera sensor model). Those are referenced and applied rather than re-derived, which is what lets the engine fit in 10 focused weeks instead of 16.
Note on AI use: The weekly lecture notes and syllabus summaries on this site are drafted with AI assistance, so each week has a consistent structure to study from. The substance is mine: every math proof and exercise solution is worked by hand on paper first and converted to LaTeX/KaTeX/MathJax using AI for typesetting only, and all code is written by me. The goal is to learn the material, which only happens by producing the proofs and the code myself.
Weekend rhythm: Saturday = 60–90 min reading + 60–90 min math/physics exercises + 2–3 hr implementation. Sunday = 2–3 hr implementation + 60 min benchmark/profiling + 60–90 min staff-level note.
General rule for each week: (1) theory exercises (applying Course 5 results), (2) implementation, (3) benchmark with recorded data, (4) staff-level note in docs/.
Goal
Build a small self-driving simulation world from scratch in modern C++20 and Vulkan.
The final project simulates: roads, lanes, intersections, traffic lights, stop signs, crosswalks, vehicles, ego vehicle, camera sensor output, depth/semantic buffers, basic vehicle physics, simple traffic rules, and self-driving perception/planning test hooks.
This is not meant to replace CARLA, Gazebo, Isaac Sim, or Unreal. The goal is a small, understandable, staff-level portfolio engine demonstrating real-time graphics architecture, physics fundamentals, simulation-world modeling, sensor simulation, road/traffic systems, planning test scenarios, C++20 systems design, Vulkan GPU programming, deterministic-ish simulation loops, and benchmark-driven engineering.
Primary tools: C++20 · Vulkan · CMake · GLFW or SDL2 · GLM · ImGui · tinyobjloader or Assimp · stb_image · GoogleTest · Tracy profiler (optional) · RenderDoc · clang-format · clang-tidy · ASan/UBSan · Python for scenario generation and analysis
Book abbreviations:
Math/Graphics: 3DMP (Dunn & Parberry) · FCG (Fundamentals of Computer Graphics — Shirley & Marschner) · MGPV (Mastering Graphics Programming with Vulkan — Castorina & Sassone) · Morin · Pressley · DPV (Dasgupta) · PR (Probabilistic Robotics) · CA (Computer Architecture) · HLW (How Linux Works) · Szeliski (Computer Vision: Algorithms and Applications)
Lectures: CS231n · MIT Machine Vision
Main repository structure:
self-driving-vulkan-sim/
docs/
engine/
core/ platform/ renderer/ assets/
physics/ simulation/ traffic/ sensors/
autonomy/ ui/
tools/
scenario_generator/ asset_pipeline/ benchmark_analysis/
tests/
shaders/ assets/ scenarios/ benchmarks/Phase 1 · Weeks 1–3 — Engine + Vulkan Foundations
Week 1 — Engine Skeleton, Windowing, and the Deterministic Main Loop
Theme: Create the engine skeleton and a deterministic, fixed-timestep simulation loop decoupled from rendering.
Read: FCG: graphics pipeline overview. · MGPV: Vulkan instance/device/swapchain overview (preview, don’t implement yet). · 3DMP (review): coordinate-system and transform conventions. (Linear maps and homogeneous transforms: assumed from Course 5.)
Week 2 — Vulkan Triangle, Swapchain, Shaders, and RenderDoc
Theme: Bring up Vulkan properly and render the first triangle.
Read: MGPV: instance, validation layers, physical/logical device, swapchain, render pass, pipeline, command buffers. · CA: GPU/CPU memory hierarchy skim.
Week 3 — Meshes, Buffers, Lighting, Materials, and Depth
Theme: Render real meshes with transforms, then make the world visually interpretable with lighting, materials, and a depth buffer.
Read: FCG: meshes, viewing, lighting, shading, materials, depth buffering. · Vulkan: vertex/index/uniform buffers, descriptor sets, depth buffers, pipeline states. · 3DMP (review): camera and transform pipeline.
Phase 2 · Weeks 4–6 — Road World, Traffic, Physics, and Collision
Week 4 — Road Geometry, Lanes, and Map Representation
Theme: Build a road/lane data model and render a procedural road network.
Read: FCG: procedural geometry. · 3DMP: splines/curves. (Curvature of curves and graph/shortest-path algorithms: assumed from Course 5.)
Week 5 — Traffic Controls, Rules, and Traffic Agents
Theme: Add traffic-control semantics (lights, stop signs, crosswalks, a rule engine) and non-ego traffic vehicles with simple car-following/lane-following behavior.
Read: DPV (apply): finite state machines, graph traversal. · PR: motion models. · Embedded AI / robotics: failsafe and state machines. · Traffic-control rules, informal.
Week 6 — Ego Vehicle Physics and Collision Detection
Theme: Add the ego vehicle with a kinematic bicycle model, then detect collisions efficiently with a broad/narrow-phase spatial structure.
Read: Morin (review): kinematics, constraints, acceleration, friction intuition. · FCG: bounding volumes, spatial data structures. · DPV: trees, hashing. · Game-physics reference (optional): rigid-body and collision basics. (Newtonian dynamics: assumed from Course 5.)
Phase 3 · Weeks 7–10 — Sensors, Perception, Planning, and Polish
Week 7 — Camera Sensor Simulation: RGB, Depth, and Semantic Ground Truth
Theme: Render from an ego-mounted camera into offscreen RGB, depth, and semantic-segmentation buffers, and export ground-truth labels.
Read: FCG: camera model, projection, render targets, framebuffers, image readback. · MIT Machine Vision: image formation, perspective projection. · CS231n: detection/segmentation skim. (Geometrical optics and image formation: assumed from Course 5.)
Week 8 — Perception Baselines: Classical + Tiny Learned Detection
Theme: Use the simulated camera output to test simple perception against ground truth.
Read: MIT Machine Vision: filtering, edges, line finding, motion vision skim. · CS231n: CNNs, detection/segmentation skim. · Szeliski: feature detection and detection basics.
Week 9 — Planning Interface, Route Following, and the Scenario System
Theme: Connect the world to a self-driving planning API with route/local planning, then build reproducible scenarios and a regression-test harness.
Read: DPV (apply): shortest paths, graph search, dynamic programming. · PR: planning under uncertainty skim. · Software-testing references. · Autonomy notes (Course 1): planner contract, prediction/planning metrics.
Week 10 — Capstone: Performance Engineering, Final Demo, and Packaging
Theme: Make the simulation scale (instancing, culling, data-oriented agent storage, multithreading), then integrate everything into a polished portfolio demo.
Read: Vulkan: instancing, descriptor management, command buffers, synchronization. · CA: parallelism, memory hierarchy, cache behavior. · HLW: processes/threads skim. · CARLA/Gazebo/Isaac-style concepts for comparison. · Waymo/Figure job-aligned system-design points.
Final Deliverables
Code: C++20 engine skeleton · Vulkan renderer · mesh loading · materials/lighting/depth · road network · lane graph · procedural road mesh · traffic lights/stop signs/crosswalks · semantic world model · traffic-rule engine · traffic vehicles · ego vehicle · kinematic bicycle model · collision detection · spatial acceleration · RGB/depth/semantic camera sensors · ground-truth exporter · classical + optional tiny-CNN detector · route + local planner · scenario system · headless runner · performance overlay · instanced rendering · culling · final demo world.
Benchmarks: Transform/math · Vulkan draw · object-count render · road mesh generation · traffic-rule query · agent simulation · vehicle physics timestep · collision broad-phase · sensor render/readback · RGB vs depth vs semantic cost · detection latency/accuracy · planning p50/p90/p99 · scenario regression results · instancing/culling · final integrated frame-time.
Portfolio Story
After 10 weeks:
I built a C++20/Vulkan self-driving simulation engine from scratch: a real-time renderer, road/lane graph, traffic controls, traffic agents, ego vehicle dynamics, collision detection, RGB/depth/semantic sensor outputs, perception baselines, route/local planning, scenario regression tests, and performance benchmarks. The project demonstrates graphics systems, physics simulation, AV simulation architecture, sensor generation, planning integration, and benchmark-driven engineering.
This complements Course 1 by providing a synthetic world to test vision/detection, trajectory prediction, planning, traffic-rule compliance, sensor simulation, failure scenarios, and latency/runtime constraints.
What to Keep Lean
Not in the first 10 weeks: full physically based rendering · full rigid-body physics engine · photorealism · OpenDRIVE importer · full CARLA clone · neural rendering · lidar raytracer · weather · pedestrian crowds · full behavior-prediction stack · full ROS integration · full editor.
Focus on: understandable engine architecture · deterministic simulation loop · Vulkan fundamentals · road/lane/traffic semantics · simple physics · useful sensor outputs · planner/perception test hooks · scenario regression tests · benchmarks and staff-level notes.