Part VII — Convex Optimization and Information Theory for ML, Signals, and Sensors

Booklet: Abridged Math Foundations for Signal Processing, Machine Learning, and Artificial Intelligence, Lessons 44–51 (PDF).

Convex sets, functions, and problems; regularized fitting and inverse problems; duality, KKT conditions, and certificates; gradient, Newton, and proximal algorithms; entropy, KL divergence, and mutual information; the asymptotic equipartition property and lossless compression; hypothesis testing, channels, and sensor information; rate–distortion, compression, and representation learning.

32 exercises across Lessons 44–51 — 17 [Hand], 15 [Proof].

Exercises

Worked sets are linked below as they are completed.

• Lesson 44 — Convex Models: Sets, Functions, and Problems — 4 exercises (2 [Hand], 2 [Proof])
• Lesson 45 — Regularized Fitting and Inverse Problems — 4 exercises (3 [Hand], 1 [Proof])
• Lesson 46 — Duality, KKT Conditions, and Certificates — 4 exercises (2 [Hand], 2 [Proof])
• Lesson 47 — Algorithms: Gradient, Newton, and Proximal Steps — 4 exercises (2 [Hand], 2 [Proof])
• Lesson 48 — Entropy, KL Divergence, and Mutual Information — 4 exercises (2 [Hand], 2 [Proof])
• Lesson 49 — The Asymptotic Equipartition Property and Lossless Compression — 4 exercises (2 [Hand], 2 [Proof])
• Lesson 50 — Testing, Channels, and Sensor Information — 4 exercises (2 [Hand], 2 [Proof])
• Lesson 51 — Rate-Distortion, Compression, and Representation Learning — 4 exercises (2 [Hand], 2 [Proof])