1.2 Unit Direction Vectors and Scalar Dot-Product Projections

Once a dataset is centered at the origin, how do we mathematically project a $d$-dimensional data vector $\mathbf{x}_i$ onto a candidate 1D line? The answer lies in the marriage of unit direction vectors and the Euclidean dot product.

In linear algebra, the dot product $\mathbf{x}_i \cdot \mathbf{u}$ measures how much two vectors point in the same direction. When $\mathbf{u}$ is normalized to unit length ($\|\mathbf{u}\| = 1$), the dot product acts as a precision laser ruler: it directly outputs the signed coordinate $z_i$ of the perpendicular shadow of $\mathbf{x}_i$ along the axis $\mathbf{u}$.

In this Article

• 1. Normalizing a Direction Vector to Unit Length
• 2. Scalar Projection: The 1D Compressed Score $z_i = \mathbf{u}^T\mathbf{x}_i$
• 3. Vector Reconstruction and the Rank-One Projector $P = \mathbf{u}\mathbf{u}^T$
• 4. Step-by-Step Worked Numerical Example
• 5. Active Recall Check and Key Takeaways

1. Normalizing a Direction Vector to Unit Length

Any nonzero vector $\mathbf{v} \in \mathbb{R}^d$ defines a line passing through the origin: $L = \{ c\mathbf{v} : c \in \mathbb{R} \}$. However, $\mathbf{v}$ and $10\mathbf{v}$ define the exact same geometric line while having different lengths.

To separate pure direction from arbitrary length, we divide $\mathbf{v}$ by its Euclidean norm $\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \dots + v_d^2}$ to obtain a unit vector $\mathbf{u}$:

$$\mathbf{u} = \frac{\mathbf{v}}{\|\mathbf{v}\|}, \quad \text{which guarantees} \quad \|\mathbf{u}\|^2 = \mathbf{u}^T \mathbf{u} = \sum_{j=1}^d u_j^2 = 1$$

Note that $\mathbf{u}$ and $-\mathbf{u}$ are the only two unit vectors spanning the line $L$; flipping $\mathbf{u}$ to $-\mathbf{u}$ simply flips the positive and negative ends of the 1D ruler without changing the line itself.

2. Scalar Projection: The 1D Compressed Score $z_i = \mathbf{u}^T\mathbf{x}_i$

Let $\theta$ be the angle between a data point $\mathbf{x}_i$ and the unit axis $\mathbf{u}$. From the geometric definition of the dot product:

$$\mathbf{x}_i \cdot \mathbf{u} = \|\mathbf{x}_i\| \, \|\mathbf{u}\| \cos\theta = \|\mathbf{x}_i\| \cos\theta \quad (\text{since } \|\mathbf{u}\| = 1)$$

In right-triangle trigonometry, $\|\mathbf{x}_i\| \cos\theta$ is literally the adjacent side of the right triangle—the signed distance from the origin to the perpendicular shadow of $\mathbf{x}_i$ on the line! In PCA terminology, this scalar value $z_i = \mathbf{u}^T \mathbf{x}_i$ is called the principal component score of observation $i$.

                      * x_i (Length = ||x_i||)
                     /|
                    / |
                   /  |  Perpendicular drop
                  /   |
                 /    |
        theta   /  _  |
      Origin (0)======(o)---------> Unit Axis u (||u|| = 1)
      |<--- z_i --->|
      z_i = ||x_i|| cos(theta) = u^T x_i

3. Vector Reconstruction and the Rank-One Projector $P = \mathbf{u}\mathbf{u}^T$

The scalar score $z_i$ is a single number telling us *how far* to walk along the line. To find the actual $d$-dimensional coordinates of that shadow point $\hat{\mathbf{x}}_i \in \mathbb{R}^d$ in the original space, we scale the unit direction vector $\mathbf{u}$ by $z_i$:

$$\hat{\mathbf{x}}_i = z_i \mathbf{u} = (\mathbf{u}^T \mathbf{x}_i)\mathbf{u} = \mathbf{u}(\mathbf{u}^T \mathbf{x}_i) = (\mathbf{u}\mathbf{u}^T)\mathbf{x}_i$$

4. Step-by-Step Worked Numerical Example

Suppose our candidate line points along $\mathbf{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$, and we want to project two centered points $\mathbf{x}_1 = \begin{bmatrix} 5 \\ 0 \end{bmatrix}$ and $\mathbf{x}_2 = \begin{bmatrix} -3 \\ -4 \end{bmatrix}$.

1. Normalize $\mathbf{v}$ to a unit vector $\mathbf{u}$: Compute $\|\mathbf{v}\| = \sqrt{3^2 + 4^2} = \sqrt{25} = 5$. Thus $\mathbf{u} = \begin{bmatrix} 0.6 \\ 0.8 \end{bmatrix}$, and we verify $0.6^2 + 0.8^2 = 0.36 + 0.64 = 1.0$.
2. Compute 1D scalar scores $z_i = \mathbf{u}^T\mathbf{x}_i$:
   • For $\mathbf{x}_1$: $z_1 = (0.6)(5) + (0.8)(0) = 3.0$.
   • For $\mathbf{x}_2$: $z_2 = (0.6)(-3) + (0.8)(-4) = -1.8 - 3.2 = -5.0$.
3. Reconstruct 2D shadow points $\hat{\mathbf{x}}_i = z_i \mathbf{u}$:
   • $\hat{\mathbf{x}}_1 = 3.0 \begin{bmatrix} 0.6 \\ 0.8 \end{bmatrix} = \begin{bmatrix} 1.8 \\ 2.4 \end{bmatrix}$.
   • $\hat{\mathbf{x}}_2 = -5.0 \begin{bmatrix} 0.6 \\ 0.8 \end{bmatrix} = \begin{bmatrix} -3.0 \\ -4.0 \end{bmatrix}$ (since $\mathbf{x}_2$ already lay on the line, its reconstruction is exact!).

5. Active Recall Check and Key Takeaways

Quick Check: Test Your Understanding

Question: Why does negating a unit eigenvector (replacing $\mathbf{u}$ with $-\mathbf{u}$) negate every scalar score $z_i$ while leaving the 2D reconstructed shadow $\hat{\mathbf{x}}_i$ completely unchanged?

Answer: Replacing $\mathbf{u}$ with $-\mathbf{u}$ changes the scalar score to $z_i^{\text{new}} = (-\mathbf{u})^T \mathbf{x}_i = -z_i$. However, when reconstructing the vector shadow, the two minus signs cancel out: $\hat{\mathbf{x}}_i^{\text{new}} = (-z_i)(-\mathbf{u}) = z_i \mathbf{u} = \hat{\mathbf{x}}_i$. Equivalently, the projection matrix $(-\mathbf{u})(-\mathbf{u})^T = \mathbf{u}\mathbf{u}^T$ is invariant to sign flips.

• Unit normalization ($\mathbf{u}^T\mathbf{u} = 1$) ensures dot products measure pure geometric distance along the projection line.
• Encoding vs. Decoding: Multiplying by $\mathbf{u}^T$ encodes $\mathbf{x}_i \in \mathbb{R}^d$ into a 1D score $z_i \in \mathbb{R}$; multiplying $z_i$ by $\mathbf{u}$ decodes it back into $\hat{\mathbf{x}}_i \in \mathbb{R}^d$.
• Canonical References: Review Wikipedia Reference on Vector Projection and MIT OpenCourseWare 18.06 Lecture on Projections onto Subspaces.

Course Navigation & Chapter Links

This article is part of the complete 24-part Synapticz public course: Intuitive Eigenvalues, Eigenvectors, and Principal Component Analysis (Top-Down & Bottom-Up).

• Chapter 1 Overview: Chapter 1: The Geometry of Data Compression — Centering, Projections, and Orthogonal Shadows
• Microlearning 1.1: 1.1 Centering Multivariate Data and Shifting the Origin to the Mean
• Microlearning 1.2: 1.2 Unit Direction Vectors and Scalar Dot-Product Projections
• Microlearning 1.3: 1.3 Orthogonal Reconstruction Error vs. Vertical Regression Residuals
• Previous Lesson: 1.1 Centering Multivariate Data and Shifting the Origin to the Mean
• Next Lesson: 1.3 Orthogonal Reconstruction Error vs. Vertical Regression Residuals