Math for Machine Learning Chapter 1: Vectors

This quiz covers fundamental concepts of vectors, distinguishing between their geometric (arrows in space) and algebraic (lists of numbers) representations. It delves into core vector operations such as vector addition and scalar multiplication, emphasizing their intuitive meaning and practical applications in Machine Learning. Understanding these concepts is paramount as vectors form the bedrock for representing data points, features, parameters, and directions of change in ML algorithms.

Key formulas and concepts include:

1. Algebraic Vector Representation:

A vector $\mathbf{v}$ in $n$-dimensional space (e.g., $\mathbb{R}^n$) is typically represented as an ordered list of $n$ numbers (components):

$\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$ or $\mathbf{v} = (v_1, v_2, \dots, v_n)$. Each $v_i$ is a real number.

2. Geometric Vector Representation:

An arrow in space, with its length representing magnitude and its orientation representing direction. A position vector starts at the origin (0,0) and points to a specific coordinate. A free vector represents displacement and can be drawn starting from any point.

3. Vector Addition:

• Algebraic: Performed component-wise. If $\mathbf{u} = (u_1, u_2, \dots, u_n)$ and $\mathbf{v} = (v_1, v_2, \dots, v_n)$, then:

  $\mathbf{u} + \mathbf{v} = (u_1+v_1, u_2+v_2, \dots, u_n+v_n)$

• Geometric:

  • Triangle Rule: Place the tail of the second vector at the head of the first. The sum is the vector from the tail of the first to the head of the second.

  • Parallelogram Rule: Place both vectors tail-to-tail. Complete the parallelogram. The sum is the diagonal starting from the common tail.

• Properties: Commutative ($\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$) and Associative ($(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$).

4. Scalar Multiplication:

• Algebraic: Multiply each component of the vector by the scalar. If $c$ is a scalar and $\mathbf{v} = (v_1, v_2, \dots, v_n)$, then:

  $c\mathbf{v} = (cv_1, cv_2, \dots, cv_n)$

• Geometric:

  • Scales the magnitude of the vector by $|c|$.

  • If $c > 0$, the direction remains the same.

  • If $c < 0$, the direction reverses (by 180 degrees).

• Properties: Associative ($(cd)\mathbf{v} = c(d\mathbf{v})$), Distributive over vector addition ($c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$), and Distributive over scalar addition ($(c+d)\mathbf{v} = c\mathbf{v} + d\mathbf{v}$).

5. Vectors in Machine Learning:

• Data Points: Features of a data instance (e.g., pixel values of an image, attributes of a customer) are represented as components of a vector.

• Parameters: Weights and biases of models (e.g., neural networks, linear regression) are often vectors.

• Directions of Change: Gradients in optimization algorithms are vectors that indicate the direction of steepest ascent (or descent).

This quiz evaluates your understanding of these core concepts, their interrelations, and their practical significance in the field of Machine Learning.