Syllabus
Course 5 — Mathematical & Theoretical Foundations · 20-Week Detailed Schedule
This course is the mathematical and theoretical backbone under the other courses. Rather than build a system, it works through the highest-value sections of the core theory textbooks and orders them by dependency: linear algebra and the analysis that makes it rigorous, then the numerical, probabilistic, and optimization machinery that everything else stands on, then complex analysis, signals, and computation, and finally the continuous physics and geometry that show up in simulation, vision, and control.
Each week lists a theme and the specific book sections it draws from. The lecture notes give definitions, key theorem statements, and intuition with worked formulas — they are study notes, not proofs. I work the proofs myself by hand.
Note on AI use: The weekly lecture notes and syllabus summaries on this site are drafted with AI assistance, so each week has a consistent structure to study from. The substance is mine: every math proof is worked by hand on paper first and converted to LaTeX/KaTeX/MathJax using AI for typesetting only. The goal is to learn the material, which only happens by producing the proofs myself.
Scope: Only the highest-value sections flagged for the applied courses are covered. Book-by-book worked solutions live in Books.
Book abbreviations:
Linear algebra, numerical & analysis: Axler (Linear Algebra Done Right) · MIR (Measure, Integration & Real Analysis, Axler) · T&B (Numerical Linear Algebra, Trefethen & Bau) · Ross (Elementary Analysis) · Mendelson (Introduction to Topology)
Probability, optimization, information: BT (Introduction to Probability, Bertsekas & Tsitsiklis) · Boyd (Convex Optimization, Boyd & Vandenberghe) · C&T (Elements of Information Theory, Cover & Thomas)
Complex analysis, signals & computation: B&N (Complex Analysis, Bak & Newman) · OWN (Signals and Systems, Oppenheim/Willsky/Nawab) · O&S (Discrete-Time Signal Processing, Oppenheim & Schafer) · DPV (Algorithms, Dasgupta/Papadimitriou/Vazirani) · Sipser (Introduction to the Theory of Computation)
Physics & geometry: Morin (Introduction to Classical Mechanics) · Pressley (Elementary Differential Geometry) · Griffiths (Introduction to Electrodynamics) · Hecht (Optics)
Algebraic structures: Pinter (A Book of Abstract Algebra)
Phase 1 · Weeks 1–5 — Linear Algebra, Numerical Computing, and Analysis Foundations
Week 1 — Linear Maps, Matrices, and Eigenstructure
Theme: The matrix as a coordinate representation of a linear map; eigenvalues, eigenvectors, upper-triangular forms, and diagonalization.
Read: Axler 3.C (matrices), 5.B (eigenvectors and upper-triangular matrices), 5.C (eigenspaces and diagonal matrices).
Week 2 — Inner Products, Orthogonality, the Spectral Theorem, and SVD
Theme: Inner product geometry, orthonormal bases and Gram–Schmidt, orthogonal projection and least squares, the spectral theorem, the SVD, and determinants.
Read: Axler 6.A–6.C (inner products, orthonormal bases, orthogonal complements and minimization), 7.B (spectral theorem), 7.D (singular value decomposition), Ch 9 (determinant-oriented material).
Week 3 — Numerical Linear Algebra I: QR, Least Squares, Conditioning, and Stability
Theme: Numbers on a real computer: norms, orthogonal projectors, QR via Gram–Schmidt and Householder, least squares, conditioning, floating-point arithmetic, and the LU factorization with pivoting.
Read: T&B Lectures 1–15 (matrix-vector products, orthogonality, norms, SVD basics, projectors, QR, Gram–Schmidt, least squares, conditioning, stability, floating point), 17–23 (Gaussian elimination, pivoting, stability of LU).
Week 4 — Numerical Linear Algebra II: Eigenvalue Algorithms and Iterative Methods
Theme: How eigenvalues and large sparse systems are actually computed: power iteration, the QR algorithm, Krylov subspaces, Arnoldi/Lanczos, conjugate gradients, and GMRES.
Read: T&B Lectures 24–29 (eigenvalue problems, power iteration, QR algorithm), 30–33 (Krylov subspaces, iterative methods), 35–40 (GMRES, conjugate gradients, Lanczos, Arnoldi).
Week 5 — Real Analysis and Metric-Space Topology
Theme: The rigor under the rest of the course: completeness, sequences and convergence, continuity, uniform convergence, differentiation, metric spaces, and compactness (the existence guarantee behind optimization).
Read: Ross Ch 1 (completeness, sup/inf), Ch 2 (sequences, Cauchy, lim sup/inf), Ch 3 (continuity, IVT/EVT), Ch 4 (uniform convergence, power series), Ch 5 (differentiation, MVT, Taylor); Mendelson Ch 2 (metric spaces), Ch 5 (compactness, Heine–Borel); MIR Ch 1–3 (Lebesgue measure, Lebesgue integration, \(L^p\) spaces — the rigorous extension above Riemann).
Phase 2 · Weeks 6–10 — Probability, Optimization, and Information
Week 6 — Probability Foundations and Random Variables
Theme: Probabilistic models, conditioning, Bayes’ rule, discrete and continuous random variables, expectation and variance, joint and conditional distributions, covariance, and transforms.
Read: BT Ch 1 (sample space and probability), Ch 2 (discrete random variables), Ch 3 (general random variables), Ch 4 (derived distributions, covariance, conditional expectation, transforms).
Week 7 — Limit Theorems, Markov Chains, and Statistical Inference
Theme: Concentration and the central limit theorem, Bernoulli/Poisson processes, discrete-time Markov chains and steady state, and both Bayesian and classical inference.
Read: BT Ch 5 (limit theorems), Ch 7 (Markov chains), Ch 8 (Bayesian inference), Ch 9 (classical inference); Ch 6 (Bernoulli and Poisson processes) for event-stream and sensor-timing intuition.
Week 8 — Convex Sets and Convex Functions
Theme: What convexity is and why it makes optimization tractable: convex sets, cones, operations preserving convexity, convex functions, conjugates, and quasiconvexity.
Read: Boyd Ch 2 (convex sets, hyperplanes, generalized inequalities), Ch 3 (convex functions, operations preserving convexity, conjugate and quasiconvex functions, log-concavity).
Week 9 — Convex Problems, Duality, and Optimization Algorithms
Theme: Standard convex problem forms, Lagrangian duality and KKT conditions, approximation and fitting, statistical estimation, and the core unconstrained/equality-constrained algorithms.
Read: Boyd Ch 4 (convex/LP/QP problems), Ch 5 (Lagrange duality, optimality conditions, sensitivity), Ch 6 (norm approximation, regularization), Ch 7 (statistical estimation), Ch 9 (gradient descent, Newton’s method), Ch 10 (equality-constrained Newton).
Week 10 — Information Theory: Entropy, Compression, Channels, and Maximum Entropy
Theme: Entropy, relative entropy and mutual information, source coding, differential entropy and the Gaussian, rate–distortion, the link to statistics, and maximum-entropy distributions.
Read: C&T Ch 2 (entropy, KL divergence, mutual information), Ch 3 (AEP), Ch 4 (entropy rates), Ch 5 (data compression), Ch 8 (differential entropy), Ch 10 (rate distortion), Ch 11 (information theory and statistics), Ch 12 (maximum entropy).
Phase 3 · Weeks 11–15 — Complex Analysis, Signals, Transforms, and Computation
Week 11 — Complex Analysis: Analytic Functions, Power Series, and Residues
Theme: Complex numbers and the plane, analytic functions and the Cauchy–Riemann equations, Cauchy’s theorem, power and Laurent series, and the residues/poles that underpin transforms and stability.
Read: B&N Ch 1 (complex numbers), Ch 2 (functions of the complex variable \(z\)), Ch 3 (analytic functions), Ch 6 (power series), Ch 7 (Laurent series, poles, and residues).
Week 12 — Signals and Systems: LTI, Fourier, and Sampling
Theme: Continuous- and discrete-time signals, LTI systems and convolution, Fourier series and transforms, frequency response, and the sampling theorem.
Read: OWN Ch 1 (signals and systems), Ch 2 (LTI systems, convolution), Ch 3 (Fourier series), Ch 4 (continuous-time Fourier transform), Ch 5 (discrete-time Fourier transform), Ch 6 (magnitude/phase, filtering), Ch 7 (sampling).
Week 13 — Transforms and DSP: Laplace, z-Transform, Filters, and the FFT
Theme: The Laplace and z-transforms, poles/zeros and stability, FIR/IIR filter design, the DFT and circular convolution, and the FFT.
Read: OWN Ch 9 (Laplace transform), Ch 10 (z-transform); O&S Ch 4 (sampling), Ch 7 (filter design), Ch 8 (DFT), Ch 9 (computation of the DFT / FFT), Ch 10 (Fourier analysis using the DFT).
Week 14 — Algorithms: Divide-and-Conquer, Graphs, Greedy, DP, and NP-Completeness
Theme: Algorithmic problem-solving paradigms and the boundary of tractability: recurrences and the FFT, graph traversal and shortest paths, greedy methods, dynamic programming, NP-completeness, and linear programming.
Read: DPV Ch 2 (divide-and-conquer, FFT), Ch 3 (graph decompositions), Ch 4 (paths in graphs), Ch 5 (greedy algorithms), Ch 6 (dynamic programming), Ch 7 (linear programming), Ch 8 (NP-complete problems), Ch 9 (coping with NP-completeness).
Week 15 — Theory of Computation: Automata, Languages, and Complexity
Theme: Formal models of computation and the complexity hierarchy: finite automata and regular languages, context-free grammars and pushdown automata, and the classes P, NP, and NP-completeness.
Read: Sipser Ch 0 (sets, functions, proof techniques), Ch 1 (regular languages, the pumping lemma), Ch 2 (context-free languages, pushdown automata), Ch 7 (time complexity, P, NP, NP-completeness).
Phase 4 · Weeks 16–18 — Continuous Physics and Geometry
Week 16 — Classical Mechanics and Lagrangian Dynamics
Theme: Newtonian dynamics, oscillations, conservation laws, angular momentum, the Lagrangian formulation, accelerating/rotating frames, and coupled oscillators / normal modes.
Read: Morin Ch 2 (\(F=ma\)), Ch 3 (oscillations), Ch 4 (energy and momentum), Ch 5 (angular momentum), Ch 6 (the Lagrangian method), Ch 8 (accelerating frames, Coriolis), Ch 10 (coupled oscillators and normal modes).
Week 17 — Differential Geometry of Curves and Surfaces
Theme: Parametrized curves, curvature and the Frenet frame, smooth surfaces and tangent planes, the first and second fundamental forms, Gaussian curvature and the Theorema Egregium, and geodesics.
Read: Pressley Ch 1 (curves, arc length, curvature, torsion), Ch 4 (surfaces, parametrizations, tangent planes), Ch 5 (first fundamental form), Ch 6 (second fundamental form, principal/Gaussian/mean curvature), Ch 8 (geodesics).
Week 18 — Electrodynamics and Optics: Fields, Waves, and Imaging
Theme: Vector calculus of fields, Maxwell’s equations and electromagnetic waves, wave motion and superposition, geometrical optics and imaging, diffraction, and Fourier optics.
Read: Griffiths Ch 1 (vector analysis), 5.1 (Lorentz force), Ch 7 (electrodynamics, Maxwell’s equations), Ch 9 (electromagnetic waves), Appendix A (curvilinear vector calculus); Hecht Ch 2 (wave motion), Ch 5–6 (geometrical optics), Ch 10 (diffraction), Ch 11 (Fourier optics).
Phase 5 · Weeks 19–20 — Algebraic Structures
Week 19 — Groups, Rings, and Fields
Theme: The three-level hierarchy of algebraic structures: groups (one operation, capturing symmetry), rings (two operations, capturing arithmetic), and fields (division rings). Subgroups, Lagrange’s theorem, homomorphisms, normal subgroups and quotient groups, ideals, and quotient rings.
Read: Pinter Ch 2–4 (operations and groups), Ch 5, 9–11 (subgroups, isomorphisms, order, cyclic groups), Ch 13–16 (cosets, Lagrange’s theorem, homomorphisms, quotient groups), Ch 17–21 (rings, ideals, quotient rings, integral domains, field of quotients).
Week 20 — Polynomial Rings, Extension Fields, and Finite Fields
Theme: \(F[x]\) as a unique factorization domain; irreducible polynomials; extension fields via \(F[x]/(p(x))\); the structure theorem for finite fields (\(\mathbb{F}_{p^k}\) is unique up to isomorphism); the Frobenius automorphism; applications to RSA, elliptic curve cryptography, and algebraic error correcting codes.
Read: Pinter Ch 22–23 (polynomial rings, factoring, irreducibles), Ch 24, 26 (extension fields, degree of extensions, tower law), Ch 28–29 (roots of polynomials, finite fields, Frobenius automorphism).
How this course relates to the other courses
- Course 1 (Autonomy ML Systems): Weeks 1–10 are the direct math core — linear algebra, analysis, numerical stability, probability, optimization, and information theory under ML and estimation.
- Course 2 (Vulkan Simulation): Weeks 16–18 (mechanics, geometry, optics) underpin vehicle physics, road/camera geometry, and sensor rendering.
- Course 3 (NLP + LLM): Weeks 6–10 (probability, optimization, information theory) and Week 14 (algorithms) are the theory behind language models, embeddings, and retrieval.
- Course 4 (Low-Level CS): Weeks 11–15 (complex analysis, signals, transforms, algorithms, computation theory) sit under DSP, performance, and complexity reasoning.
- Course 6 (Microelectronic Circuits & Signal Processing): Weeks 11–15 (complex analysis, signals and transforms, the DFT/FFT) are the pure-theory counterpart to Course 6’s applied phasors, Fourier analysis, sampling, and digital filters; Weeks 16–18 (electrodynamics/optics) sit under its field-to-circuit foundations.
- Weeks 19–20 (Algebraic Structures): Abstract algebra and finite fields fall outside the scope of the applied courses but underpin cryptographic systems, forward error correction, and algebraic coding theory.