Part VIII — Convex Optimization and Information Theory for ML, Signals, and Sensors

Booklet: Abridged Math Foundations for Signals and Systems, Lessons 53–60 (PDF).

Convex sets, functions, and problems; regularized fitting and inverse problems; duality, KKT conditions, and certificates; gradient, Newton, and proximal algorithms; entropy, KL divergence, and mutual information; hypothesis testing, channels, and sensor information; rate–distortion, compression, and representation learning.

28 exercises across Lessons 53–59 — 15 [Hand], 13 [Proof]. Lesson 60 is Capstone VIII (a sensor pipeline optimized under an information budget, no exercise block).

Exercises

Worked sets are linked below as they are completed.

  • Lesson 53 — Convex Models: Sets, Functions, and Problems — 4 exercises (2 [Hand], 2 [Proof])
  • Lesson 54 — Regularized Fitting and Inverse Problems — 4 exercises (3 [Hand], 1 [Proof])
  • Lesson 55 — Duality, KKT Conditions, and Certificates — 4 exercises (2 [Hand], 2 [Proof])
  • Lesson 56 — Algorithms: Gradient, Newton, and Proximal Steps — 4 exercises (2 [Hand], 2 [Proof])
  • Lesson 57 — Entropy, KL Divergence, and Mutual Information — 4 exercises (2 [Hand], 2 [Proof])
  • Lesson 58 — Testing, Channels, and Sensor Information — 4 exercises (2 [Hand], 2 [Proof])
  • Lesson 59 — Rate-Distortion, Compression, and Representation Learning — 4 exercises (2 [Hand], 2 [Proof])
  • Lesson 60 — Capstone VIII: Optimize a Sensor Pipeline Under an Information Budget — project