Math for Machine Learning - Chapter 10: Fundamental Subspaces (Column space, Row space and Null space)

This advanced quiz delves into the foundational concepts of linear algebra crucial for machine learning: the four fundamental subspaces of a matrix. Understanding the Column Space, Row Space, Null Space, and Left Null Space is paramount for grasping how linear transformations behave, the solvability of linear systems, and the underlying geometry of data. This quiz challenges your comprehension of their definitions, properties, bases, dimensions, and intricate orthogonal relationships. Key concepts include:

• The Column Space $C(A)$: The span of the columns of $A$, representing the set of all possible outputs $Ax$.

• The Row Space $R(A)$: The span of the rows of $A$, which is equivalent to $C(A^T)$.

• The Null Space $N(A)$: The set of all vectors $x$ such that $Ax = 0$.

• The Left Null Space $N(A^T)$: The set of all vectors $y$ such that $A^Ty = 0$, which is the null space of $A^T$.

• Rank-Nullity Theorem: For an $m \times n$ matrix $A$, $rank(A) + nullity(A) = n$.

• Fundamental Theorem of Linear Algebra: The Row Space and Null Space are orthogonal complements ($R(A) = N(A)^\perp$), and the Column Space and Left Null Space are orthogonal complements ($C(A) = N(A^T)^\perp$).

• Orthogonal Projection: The projection of a vector $b$ onto a subspace $S$ with basis columns $U$ is given by $P_S b$, where $P_S = U(U^T U)^{-1} U^T$. If $U$ contains an orthonormal basis for $S$, $P_S = U U^T$.

This quiz will push your analytical skills to master these theoretical underpinnings, essential for tasks like dimensionality reduction, understanding model capacities, and optimizing algorithms in machine learning.