Unlocking Matrix Transformations: A Guide to Eigenvectors and Eigenvalues

In the vast world of linear algebra, eigenvectors and eigenvalues stand out as fundamental concepts with profound implications. They are the keys to understanding the true nature of linear transformations, revealing which directions remain unchanged. From quantum mechanics to Google's PageRank algorithm, their applications are everywhere. This guide will demystify their calculation, providing a clear, step-by-step process complemented by numerous solved problems.

The Fundamental Eigenvalue Equation

The relationship between a matrix `A`, its eigenvector `v`, and its eigenvalue `λ` is elegantly captured in a single equation:

This equation states that when the matrix `A` acts on (transforms) its eigenvector `v`, the result is simply the original vector `v` scaled by a constant factor, the eigenvalue `λ`. The eigenvector `v` must be a non-zero vector, otherwise, the equation would hold true for any matrix and any scalar `λ`.

The 3-Step Calculation Process

To find the eigenvalues and eigenvectors of a square matrix `A`, we follow a systematic, three-step procedure.

Solved Problems

Let's solidify our understanding by working through several examples, starting with 2x2 matrices and moving to 3x3 matrices.

Problem 1: A Simple 2x2 Matrix

Find the eigenvalues and eigenvectors of the matrix: $$ A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} $$

Step 1: Characteristic Equation

$$ \det(A - \lambda I) = \det\left(\begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\right) = 0 $$

$$ \det\begin{pmatrix} 4-\lambda & 1 \\ 2 & 3-\lambda \end{pmatrix} = 0 $$

Step 2: Solve for Eigenvalues (λ)

$$ (4-\lambda)(3-\lambda) - (1)(2) = 0 $$

$$ 12 - 4\lambda - 3\lambda + \lambda^2 - 2 = 0 $$

$$ \lambda^2 - 7\lambda + 10 = 0 $$

$$ (\lambda - 5)(\lambda - 2) = 0 $$

The eigenvalues are $$ \lambda_1 = 5 $$ and $$ \lambda_2 = 2 $$.

Step 3: Find Eigenvectors (v)

For $$ \lambda_1 = 5 $$:

We solve $$ (A - 5I)v = 0 $$:

$$ \begin{pmatrix} 4-5 & 1 \\ 2 & 3-5 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $$

This gives the equation $$ -x + y = 0 $$, which means $$ x = y $$. A simple choice is $$ x=1, y=1 $$. So, the eigenvector is $$ v_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix} $$.

For $$ \lambda_2 = 2 $$:

We solve $$ (A - 2I)v = 0 $$:

$$ \begin{pmatrix} 4-2 & 1 \\ 2 & 3-2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $$

This gives the equation $$ 2x + y = 0 $$, which means $$ y = -2x $$. A simple choice is $$ x=1, y=-2 $$. So, the eigenvector is $$ v_2 = \begin{pmatrix} 1 \\ -2 \end{pmatrix} $$.

Problem 2: A Matrix with a Zero Eigenvalue

Find the eigenvalues and eigenvectors of the matrix: $$ A = \begin{pmatrix} 1 & 2 \\ 1 & 2 \end{pmatrix} $$

Step 1 & 2: Find Eigenvalues

$$ \det\begin{pmatrix} 1-\lambda & 2 \\ 1 & 2-\lambda \end{pmatrix} = (1-\lambda)(2-\lambda) - 2 = 0 $$

$$ 2 - \lambda - 2\lambda + \lambda^2 - 2 = 0 \implies \lambda^2 - 3\lambda = 0 $$

$$ \lambda(\lambda - 3) = 0 $$

The eigenvalues are $$ \lambda_1 = 3 $$ and $$ \lambda_2 = 0 $$.

Step 3: Find Eigenvectors

For $$ \lambda_1 = 3 $$: $$ (A - 3I)v = 0 $$

$$ \begin{pmatrix} -2 & 2 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies -2x + 2y = 0 \implies x=y $$. Let $$ x=1 $$, then $$ v_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix} $$.

For $$ \lambda_2 = 0 $$: $$ (A - 0I)v = 0 $$

$$ \begin{pmatrix} 1 & 2 \\ 1 & 2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies x + 2y = 0 \implies x=-2y $$. Let $$ y=1 $$, then $$ v_2 = \begin{pmatrix} -2 \\ 1 \end{pmatrix} $$.

Problem 3: Repeated Eigenvalues

Find the eigenvalues and eigenvectors of the matrix: $$ A = \begin{pmatrix} 2 & 1 \\ 0 & 2 \end{pmatrix} $$

Step 1 & 2: Find Eigenvalues

$$ \det\begin{pmatrix} 2-\lambda & 1 \\ 0 & 2-\lambda \end{pmatrix} = (2-\lambda)(2-\lambda) - 0 = 0 $$

$$ (\lambda - 2)^2 = 0 $$

We have a repeated eigenvalue: $$ \lambda_1 = \lambda_2 = 2 $$.

Step 3: Find Eigenvectors

For $$ \lambda = 2 $$: $$ (A - 2I)v = 0 $$

$$ \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies 0x + 1y = 0 \implies y=0 $$.

`x` can be any non-zero value. Let's choose `x=1`. The eigenvector is $$ v = \begin{pmatrix} 1 \\ 0 \end{pmatrix} $$. In this case, even though the eigenvalue is repeated, we can only find one linearly independent eigenvector. This means the matrix is not diagonalizable.

Problem 4: Complex Eigenvalues

Find the eigenvalues and eigenvectors of the matrix: $$ A = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} $$ (A 90-degree counter-clockwise rotation matrix)

Step 1 & 2: Find Eigenvalues

$$ \det\begin{pmatrix} -\lambda & -1 \\ 1 & -\lambda \end{pmatrix} = (-\lambda)(-\lambda) - (-1)(1) = 0 $$

$$ \lambda^2 + 1 = 0 \implies \lambda^2 = -1 $$

The eigenvalues are complex: $$ \lambda_1 = i $$ and $$ \lambda_2 = -i $$.

Step 3: Find Eigenvectors

For $$ \lambda_1 = i $$: $$ (A - iI)v = 0 $$

$$ \begin{pmatrix} -i & -1 \\ 1 & -i \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies -ix - y = 0 \implies y = -ix $$. Let $$ x=1 $$, then $$ v_1 = \begin{pmatrix} 1 \\ -i \end{pmatrix} $$.

For $$ \lambda_2 = -i $$: $$ (A + iI)v = 0 $$

$$ \begin{pmatrix} i & -1 \\ 1 & i \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies ix - y = 0 \implies y = ix $$. Let $$ x=1 $$, then $$ v_2 = \begin{pmatrix} 1 \\ i \end{pmatrix} $$.

Geometrically, this makes sense. A 90-degree rotation in a real 2D plane changes the direction of every vector. There are no real eigenvectors. The eigenvectors exist in a complex vector space.

Problem 5: A Simple 3x3 Matrix

Find the eigenvalues and eigenvectors of: $$ A = \begin{pmatrix} 1 & -1 & 0 \\ 0 & 2 & 1 \\ 0 & 0 & 3 \end{pmatrix} $$

Step 1 & 2: Find Eigenvalues

This is an upper triangular matrix. The determinant of a triangular matrix is the product of its diagonal entries.

$$ \det(A - \lambda I) = \det\begin{pmatrix} 1-\lambda & -1 & 0 \\ 0 & 2-\lambda & 1 \\ 0 & 0 & 3-\lambda \end{pmatrix} = (1-\lambda)(2-\lambda)(3-\lambda) = 0 $$

The eigenvalues are the diagonal entries: $$ \lambda_1 = 1, \lambda_2 = 2, \lambda_3 = 3 $$.

Step 3: Find Eigenvectors

For $$ \lambda_1 = 1 $$: Solve $$ (A-I)v=0 $$

$$ \begin{pmatrix} 0 & -1 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 2 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. This gives $$ -y=0 \implies y=0 $$ and $$ y+z=0 \implies z=0 $$. `x` can be anything. Let $$ x=1 $$. So, $$ v_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} $$.

For $$ \lambda_2 = 2 $$: Solve $$ (A-2I)v=0 $$

$$ \begin{pmatrix} -1 & -1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. This gives $$ z=0 $$ and $$ -x-y=0 \implies x=-y $$. Let $$ y=1 $$, then $$ x=-1 $$. So, $$ v_2 = \begin{pmatrix} -1 \\ 1 \\ 0 \end{pmatrix} $$.

For $$ \lambda_3 = 3 $$: Solve $$ (A-3I)v=0 $$

$$ \begin{pmatrix} -2 & -1 & 0 \\ 0 & -1 & 1 \\ 0 & 0 & 0 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. This gives $$ -y+z=0 \implies y=z $$. Also, $$ -2x-y=0 \implies y=-2x $$. Let $$ x=1 $$, then $$ y=-2 $$ and $$ z=-2 $$. So, $$ v_3 = \begin{pmatrix} 1 \\ -2 \\ -2 \end{pmatrix} $$.

Problem 6: 3x3 with Repeated Eigenvalue

Find the eigenvalues and eigenvectors of: $$ A = \begin{pmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

This is also triangular. The eigenvalues are $$ \lambda_1 = 1, \lambda_2 = 2, \lambda_3 = 2 $$. We have a repeated eigenvalue.

For $$ \lambda_1 = 1 $$: Solve $$ (A-I)v=0 $$

$$ \begin{pmatrix} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. This gives $$ y=0 $$ and $$ x+y=0 \implies x=0 $$. `z` can be anything. Let $$ z=1 $$. So, $$ v_1 = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} $$.

For $$ \lambda = 2 $$ (repeated): Solve $$ (A-2I)v=0 $$

$$ \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & -1 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. This gives $$ y=0 $$ and $$ -z=0 \implies z=0 $$. `x` can be anything. Let $$ x=1 $$. So, the eigenvector is $$ v_2 = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} $$. Again, we only find one linearly independent eigenvector for the repeated eigenvalue.

Problem 7: A Standard 3x3 Matrix

Find the eigenvalues and eigenvectors of: $$ A = \begin{pmatrix} 0 & 0 & -2 \\ 1 & 2 & 1 \\ 1 & 0 & 3 \end{pmatrix} $$

Step 1 & 2: Find Eigenvalues

$$ \det\begin{pmatrix} -\lambda & 0 & -2 \\ 1 & 2-\lambda & 1 \\ 1 & 0 & 3-\lambda \end{pmatrix} = 0 $$

Using cofactor expansion along the second column:

$$ (2-\lambda) \det\begin{pmatrix} -\lambda & -2 \\ 1 & 3-\lambda \end{pmatrix} = (2-\lambda)(-\lambda(3-\lambda) - (-2)(1)) = 0 $$

$$ (2-\lambda)(-3\lambda + \lambda^2 + 2) = (2-\lambda)(\lambda^2 - 3\lambda + 2) = 0 $$

$$ (2-\lambda)(\lambda - 1)(\lambda - 2) = 0 $$

The eigenvalues are $$ \lambda_1 = 1, \lambda_2 = 2, \lambda_3 = 2 $$. Another repeated eigenvalue.

Step 3: Find Eigenvectors

For $$ \lambda_1 = 1 $$: Solve $$ (A-I)v=0 $$

$$ \begin{pmatrix} -1 & 0 & -2 \\ 1 & 1 & 1 \\ 1 & 0 & 2 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. First row: $$ -x - 2z = 0 \implies x = -2z $$. Second row: $$ x+y+z=0 $$. Substituting x: $$ -2z+y+z=0 \implies y=z $$. Let $$ z=1 $$, then $$ x=-2, y=1 $$. So, $$ v_1 = \begin{pmatrix} -2 \\ 1 \\ 1 \end{pmatrix} $$.

For $$ \lambda = 2 $$ (repeated): Solve $$ (A-2I)v=0 $$

$$ \begin{pmatrix} -2 & 0 & -2 \\ 1 & 0 & 1 \\ 1 & 0 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix} $$. All rows give the same equation: $$ x+z=0 \implies x=-z $$. `y` is not constrained, it can be any value. This means we can find two linearly independent eigenvectors!

1. Let $$ y=1 $$ and $$ z=0 $$. Then $$ x=0 $$. So, $$ v_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} $$.

2. Let $$ y=0 $$ and $$ z=1 $$. Then $$ x=-1 $$. So, $$ v_3 = \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} $$.

In this case, the repeated eigenvalue $$ \lambda=2 $$ has an eigenspace of dimension 2. The matrix is diagonalizable.

Problem 8: A Symmetric Matrix

Find the eigenvalues and eigenvectors of: $$ A = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} $$

Step 1 & 2: Find Eigenvalues

$$ \det\begin{pmatrix} 1-\lambda & 2 \\ 2 & 1-\lambda \end{pmatrix} = (1-\lambda)^2 - 4 = 0 $$

$$ (1-\lambda)^2 = 4 \implies 1-\lambda = \pm 2 $$

$$ \lambda = 1 \mp 2 $$. So, $$ \lambda_1 = -1 $$ and $$ \lambda_2 = 3 $$. (Note: Real eigenvalues, as expected for a symmetric matrix).

Step 3: Find Eigenvectors

For $$ \lambda_1 = -1 $$: $$ (A - (-1)I)v = 0 $$

$$ \begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies 2x + 2y = 0 \implies x = -y $$. Let $$ y=1 $$, then $$ v_1 = \begin{pmatrix} -1 \\ 1 \end{pmatrix} $$.

For $$ \lambda_2 = 3 $$: $$ (A - 3I)v = 0 $$

$$ \begin{pmatrix} -2 & 2 \\ 2 & -2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies -2x + 2y = 0 \implies x = y $$. Let $$ y=1 $$, then $$ v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix} $$.

Problem 9: A Diagonal Matrix

Find the eigenvalues and eigenvectors of: $$ A = \begin{pmatrix} 7 & 0 & 0 \\ 0 & -2 & 0 \\ 0 & 0 & 5 \end{pmatrix} $$

This is a special case of a triangular matrix. The eigenvalues are simply the diagonal entries: $$ \lambda_1 = 7, \lambda_2 = -2, \lambda_3 = 5 $$.

The eigenvectors are the standard basis vectors:

For $$ \lambda_1=7 $$, the eigenvector is $$ v_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} $$.

For $$ \lambda_2=-2 $$, the eigenvector is $$ v_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} $$.

For $$ \lambda_3=5 $$, the eigenvector is $$ v_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} $$.

Problem 10: Another 3x3 Example

Find the eigenvalues and eigenvectors of: $$ A = \begin{pmatrix} 1 & 1 & 2 \\ 1 & 2 & 1 \\ 2 & 1 & 1 \end{pmatrix} $$

Step 1 & 2: Find Eigenvalues

$$ \det\begin{pmatrix} 1-\lambda & 1 & 2 \\ 1 & 2-\lambda & 1 \\ 2 & 1 & 1-\lambda \end{pmatrix} = 0 $$

$$ (1-\lambda)[(2-\lambda)(1-\lambda)-1] - 1[(1-\lambda)-2] + 2[1-2(2-\lambda)] = 0 $$

$$ (1-\lambda)[2 - 3\lambda + \lambda^2 - 1] - [-\lambda - 1] + 2[1 - 4 + 2\lambda] = 0 $$

$$ (1-\lambda)(\lambda^2 - 3\lambda + 1) + \lambda + 1 + 2(2\lambda - 3) = 0 $$

$$ \lambda^2 - 3\lambda + 1 - \lambda^3 + 3\lambda^2 - \lambda + \lambda + 1 + 4\lambda - 6 = 0 $$

$$ -\lambda^3 + 4\lambda^2 + \lambda - 4 = 0 \implies \lambda^3 - 4\lambda^2 - \lambda + 4 = 0 $$

$$ \lambda^2(\lambda - 4) - 1(\lambda - 4) = 0 \implies (\lambda^2 - 1)(\lambda - 4) = 0 $$

$$ (\lambda-1)(\lambda+1)(\lambda-4) = 0 $$

The eigenvalues are $$ \lambda_1 = 4, \lambda_2 = 1, \lambda_3 = -1 $$.

Step 3: Find Eigenvectors

For $$ \lambda_1 = 4 $$: $$ \begin{pmatrix} -3 & 1 & 2 \\ 1 & -2 & 1 \\ 2 & 1 & -3 \end{pmatrix}v = 0 $$. Solving gives $$ v_1 = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} $$.

For $$ \lambda_2 = 1 $$: $$ \begin{pmatrix} 0 & 1 & 2 \\ 1 & 1 & 1 \\ 2 & 1 & 0 \end{pmatrix}v = 0 $$. Solving gives $$ v_2 = \begin{pmatrix} 1 \\ -2 \\ 1 \end{pmatrix} $$.

For $$ \lambda_3 = -1 $$: $$ \begin{pmatrix} 2 & 1 & 2 \\ 1 & 3 & 1 \\ 2 & 1 & 2 \end{pmatrix}v = 0 $$. Solving gives $$ v_3 = \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} $$.

Conclusion

Calculating eigenvalues and eigenvectors is a procedural task that unlocks deep insights into the behavior of matrices. By consistently applying the three-step process—forming the characteristic equation, solving for eigenvalues, and then solving for the corresponding eigenvectors—you can analyze any square matrix. These special vectors and their scalar multipliers are not just abstract mathematical curiosities; they are essential tools in physics, engineering, computer science, and data analysis, providing a powerful way to simplify and understand complex systems.