Math for Machine Learning - Chapter 5: Norms and Distance Metrics

This advanced quiz is designed for those who wish to achieve mastery over the concepts of vector norms and distance metrics. It moves beyond simple calculations to challenge your understanding of the deep geometric, algebraic, and analytical properties of L1, L2, and L-infinity norms. Questions will probe the intricate relationships between different norms, their impact on optimization problems (like inducing sparsity), the geometry of their unit balls, and their failure cases. You will encounter problems involving induced matrix norms, norms on function spaces, and theoretical concepts like the parallelogram law and Hölder's inequality. Prepare to apply your knowledge to solve complex problems that require a synthesis of multiple concepts. This is not just a test of memory, but a rigorous assessment of your ability to reason about vector spaces and their measures.

Key Formulas:

• Lp Norm: $||x||_p = \left( \sum_{i=1}^{n} |x_i|^p \right)^{1/p}$

• L1 Norm (Manhattan): $||x||_1 = \sum_{i=1}^{n} |x_i|$

• L2 Norm (Euclidean): $||x||_2 = \sqrt{\sum_{i=1}^{n} x_i^2}$

• L-infinity Norm (Max): $||x||_\infty = \max_{i} |x_i|$

• p-Distance: $d_p(x, y) = ||x - y||_p$

• Hölder's Inequality: $|x^T y| \le ||x||_p ||y||_q$ for $\frac{1}{p} + \frac{1}{q} = 1$