1.1 Centering Multivariate Data and Shifting the Origin to the Mean

Every Principal Component Analysis pipeline begins with an essential coordinate transformation: mean-centering. Before searching for directions of maximum spread or minimum reconstruction error, we must shift the coordinate axes so that the origin $\mathbf{0}$ sits directly at the empirical center of mass (the sample mean vector $\boldsymbol{\mu}$) of the dataset.

Why is centering non-negotiable? Because any vector subspace—whether a 1D line $\text{span}(\mathbf{u})$ or a $k$-dimensional hyperplane—is mathematically locked to pass through the origin $\mathbf{0}$. If the data cloud sits far away from $\mathbf{0}$, a line through the origin cannot pass through the middle of the cloud unless it points toward $\boldsymbol{\mu}$, conflating the average location of the data with its internal directional variance.

In this Article

• 1. Computing the Sample Mean Vector and Centered Data Matrix
• 2. Why Translation Preserves Pairwise Distances and Angles
• 3. The Uncentered Trap: Phantom First Principal Components
• 4. Step-by-Step Worked Numerical Example
• 5. Active Recall Check and Key Takeaways

1. Computing the Sample Mean Vector and Centered Data Matrix

Let $X_{\text{raw}} \in \mathbb{R}^{n \times d}$ be a data matrix whose $n$ rows are observations (samples) and whose $d$ columns are features (variables). Denote the $i$-th observation as a column vector $\mathbf{x}_i^{\text{raw}} \in \mathbb{R}^d$.

The sample mean vector $\boldsymbol{\mu} \in \mathbb{R}^d$ averages each feature column across all $n$ observations:

$$\boldsymbol{\mu} = \frac{1}{n} \sum_{i=1}^n \mathbf{x}_i^{\text{raw}} = \frac{1}{n} X_{\text{raw}}^T \mathbf{1}_n$$

where $\mathbf{1}_n = \begin{bmatrix} 1 & 1 & \dots & 1 \end{bmatrix}^T \in \mathbb{R}^n$ is the all-ones column vector. Subtracting $\boldsymbol{\mu}$ from each observation produces the centered vector $\mathbf{x}_i = \mathbf{x}_i^{\text{raw}} - \boldsymbol{\mu}$. In compact matrix form, using the centering matrix $H = I_n - \frac{1}{n}\mathbf{1}_n \mathbf{1}_n^T$:

$$X = X_{\text{raw}} - \mathbf{1}_n \boldsymbol{\mu}^T = H X_{\text{raw}}$$

2. Why Translation Preserves Pairwise Distances and Angles

Does subtracting $\boldsymbol{\mu}$ distort the internal geometry of our dataset? Not at all. Consider the displacement vector between any two observations $i$ and $k$ after centering:

$$\mathbf{x}_i - \mathbf{x}_k = (\mathbf{x}_i^{\text{raw}} - \boldsymbol{\mu}) - (\mathbf{x}_k^{\text{raw}} - \boldsymbol{\mu}) = \mathbf{x}_i^{\text{raw}} - \mathbf{x}_k^{\text{raw}}$$

Because $\boldsymbol{\mu}$ cancels out identically, every pairwise Euclidean distance $\|\mathbf{x}_i - \mathbf{x}_k\|$, every internal angle between pairs of points, and the shape of the cloud's convex hull remain 100% invariant. We have not deformed the cloud; we have merely slid our coordinate crosshairs to its center of gravity.

   BEFORE CENTERING (Origin far from cloud)        AFTER CENTERING (Origin at centroid)
   x2_raw                                          x2
    ^              *  *                             ^         *  *
    |           *  M  *   (Cloud at M=[10,10])      |      *  |  *
    |           *  *                                +------*--0--*------> x1
    |      /                                        |      *  |
    |    /  <-- Line through (0,0) forced           |
    +--0---------------------> x1_raw               (Line through 0 now slices
    (Dominated by vector 0 -> M!)                    along the cloud's true axis!)

3. The Uncentered Trap: Phantom First Principal Components

Suppose a tight horizontal cloud of points sits around $\boldsymbol{\mu} = \begin{bmatrix} 100 \\ 100 \end{bmatrix}$ with horizontal spread $\pm 2$ and vertical spread $\pm 0.1$. Its true direction of variation is purely horizontal ($\begin{bmatrix} 1 \\ 0 \end{bmatrix}$).

However, if you forget to subtract $\boldsymbol{\mu}$ and fit a line through the origin $\begin{bmatrix} 0 \\ 0 \end{bmatrix}$, any horizontal line through $\begin{bmatrix} 0 \\ 0 \end{bmatrix}$ misses the cloud by $100$ vertical units! To get near the points at $\begin{bmatrix} 100 \\ 100 \end{bmatrix}$, the uncentered algorithm is forced to tilt the line to $45^\circ$ ($\begin{bmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \end{bmatrix}$), reporting a completely bogus direction that reflects the mean offset $\boldsymbol{\mu}$ rather than the horizontal variance.

4. Step-by-Step Worked Numerical Example

Let us center a 3-observation, 2-feature dataset $X_{\text{raw}}$:

Observation $i$	Raw Feature 1 ($x_{i1}^{\text{raw}}$)	Raw Feature 2 ($x_{i2}^{\text{raw}}$)	Centered $x_{i1} = x_{i1}^{\text{raw}} - 4$	Centered $x_{i2} = x_{i2}^{\text{raw}} - 6$
$$i = 1$$	$$2$$	$$3$$	$$-2$$	$$-3$$
$$i = 2$$	$$4$$	$$6$$	$$0$$	$$0$$
$$i = 3$$	$$6$$	$$9$$	$$+2$$	$$+3$$
Column Mean	$$\mu_1 = \frac{2+4+6}{3} = 4$$	$$\mu_2 = \frac{3+6+9}{3} = 6$$	$$\frac{-2+0+2}{3} = 0$$	$$\frac{-3+0+3}{3} = 0$$

Notice how the centered points $\begin{bmatrix} -2 \\ -3 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ 3 \end{bmatrix}$ now lie cleanly on a single line through the origin with direction vector $\begin{bmatrix} 2 \\ 3 \end{bmatrix}$.

5. Active Recall Check and Key Takeaways

Quick Check: Test Your Understanding

Question: Given three 2D observations $\mathbf{x}_1^{\text{raw}} = \begin{bmatrix} 10 \\ 2 \end{bmatrix}$, $\mathbf{x}_2^{\text{raw}} = \begin{bmatrix} 12 \\ 5 \end{bmatrix}$, and $\mathbf{x}_3^{\text{raw}} = \begin{bmatrix} 14 \\ 8 \end{bmatrix}$, compute the mean vector $\boldsymbol{\mu}$ and the centered vector $\mathbf{x}_1$. What is the Euclidean distance between observations 1 and 3 before and after centering?

Answer: The mean vector is $\boldsymbol{\mu} = \frac{1}{3}\begin{bmatrix} 10+12+14 \\ 2+5+8 \end{bmatrix} = \begin{bmatrix} 12 \\ 5 \end{bmatrix}$. Subtracting $\boldsymbol{\mu}$ from $\mathbf{x}_1^{\text{raw}}$ gives $\mathbf{x}_1 = \begin{bmatrix} 10-12 \\ 2-5 \end{bmatrix} = \begin{bmatrix} -2 \\ -3 \end{bmatrix}$ (and $\mathbf{x}_3 = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$). Before centering, the displacement is $\begin{bmatrix} 14-10 \\ 8-2 \end{bmatrix} = \begin{bmatrix} 4 \\ 6 \end{bmatrix}$ with distance $\sqrt{4^2+6^2} = \sqrt{52}$. After centering, the displacement is $\begin{bmatrix} 2-(-2) \\ 3-(-3) \end{bmatrix} = \begin{bmatrix} 4 \\ 6 \end{bmatrix}$ with the exact same distance $\sqrt{52}$.

• Subspaces pass through $\mathbf{0}$: Centering aligns the origin of our vector space with the centroid $\boldsymbol{\mu}$ of the data.
• Rigid translation: Subtracting $\boldsymbol{\mu}$ leaves all pairwise distances, angles, and variances completely unchanged.
• Canonical References: See Scikit-Learn User Guide on Exact PCA and Mean Centering and Wikipedia Reference on the Centering Matrix.

Course Navigation & Chapter Links

This article is part of the complete 24-part Synapticz public course: Intuitive Eigenvalues, Eigenvectors, and Principal Component Analysis (Top-Down & Bottom-Up).

• Chapter 1 Overview: Chapter 1: The Geometry of Data Compression — Centering, Projections, and Orthogonal Shadows
• Microlearning 1.1: 1.1 Centering Multivariate Data and Shifting the Origin to the Mean
• Microlearning 1.2: 1.2 Unit Direction Vectors and Scalar Dot-Product Projections
• Microlearning 1.3: 1.3 Orthogonal Reconstruction Error vs. Vertical Regression Residuals
• Previous Lesson: Chapter 1: The Geometry of Data Compression — Centering, Projections, and Orthogonal Shadows
• Next Lesson: 1.2 Unit Direction Vectors and Scalar Dot-Product Projections