# Eigenvectors & Eigenvalues

> An eigenvector keeps its direction under A and is only scaled: Av = λv.

Source: https://www.mlfoundry.in/learn/eigenvectors-eigenvalues  
Section: Mathematics › Linear Algebra  
Level: core · 6 min read · updated 2026-10-03

**An eigenvector:** of a square matrix $A$ is a nonzero vector $v$ that $A$ maps onto its own line: $Av=\lambda v$. The scalar $\lambda$ is the corresponding **eigenvalue** — the factor by which that direction is stretched (negative $\lambda$ flips it).

- **Notation:** $Av=\lambda v$; $A=V\Lambda V^{-1}$ with eigenvectors as columns of $V$ and $\Lambda=\operatorname{diag}(\lambda_1,\dots,\lambda_n)$
- **Input → output:** a square $n\times n$ matrix → $n$ eigenvalues (with multiplicity, possibly complex) and their eigenvectors
- **Key property:** $\operatorname{tr}A=\sum_i\lambda_i$, $\det A=\prod_i\lambda_i$; symmetric $A$ has real $\lambda$ and orthonormal eigenvectors
- **Prerequisites:** matrix multiplication, determinant, linear systems & rank
- **Used in:** PCA, PageRank, spectral clustering, Markov chains, curvature of loss surfaces

## The intuition

A matrix moves almost every vector off its own line: it turns the vector as well as stretching it. Eigenvectors are the exceptions — the directions the map leaves pointing the same way. Along them, a complicated matrix acts like a plain number.

That is why they matter. If you write any vector in eigenvector coordinates, applying $A$ just multiplies each coordinate by its own $\lambda$, and applying $A$ a hundred times multiplies by $\lambda^{100}$. Directions with $|\lambda|>1$ blow up, those with $|\lambda|<1$ die out, and the largest $|\lambda|$ eventually dominates everything.

## The math

$Av=\lambda v$ means $(A-\lambda I)v=0$ has a nonzero solution, i.e. $A-\lambda I$ is singular. So the eigenvalues are the roots of the **characteristic polynomial**, and each eigenvector spans part of the null space of $A-\lambda I$:

$$
\det(A-\lambda I)=0,\qquad (A-\lambda I)\,v=0,\ v\ne0
$$

- $I$: identity matrix of the same size as A
- $\lambda$: an eigenvalue — a root of the degree-n polynomial

For a $2\times2$ matrix the polynomial is $\lambda^2-(\operatorname{tr}A)\lambda+\det A$, so

$$
\lambda_{1,2}=\frac{\operatorname{tr}A}{2}\pm\sqrt{\frac{(\operatorname{tr}A)^2}{4}-\det A}
$$

- $\operatorname{tr}A$: trace, $a_{11}+a_{22}$
- $\det A$: $a_{11}a_{22}-a_{12}a_{21}$

A negative discriminant gives a complex pair: a rotation has no real direction it preserves. A zero discriminant gives a repeated eigenvalue, which may come with only one eigenvector — the shear $\begin{pmatrix}1&1\\0&1\end{pmatrix}$ has $\lambda=1$ twice but only the direction $(1,0)$.

**Diagonalization.** If $A$ has $n$ independent eigenvectors, stack them as the columns of $V$: then $AV=V\Lambda$, so $A=V\Lambda V^{-1}$ and powers become cheap, $A^k=V\Lambda^kV^{-1}$. For symmetric $A$ the **spectral theorem** guarantees real eigenvalues and an orthonormal $V=Q$:

$$
A=Q\Lambda Q^\top=\sum_{i=1}^n\lambda_i\,q_iq_i^\top,\qquad \lambda_{\max}=\max_{\lVert x\rVert=1}x^\top Ax
$$

- $q_i$: orthonormal eigenvectors (columns of Q)
- $x^\top Ax$: Rayleigh quotient of a unit vector x

**Power iteration.** Write $x_0=\sum_i c_iv_i$. Then $A^kx_0=\sum_i c_i\lambda_i^kv_i=\lambda_1^k\big(c_1v_1+\sum_{i>1}c_i(\lambda_i/\lambda_1)^kv_i\big)$. If $|\lambda_1|>|\lambda_2|$ and $c_1\ne0$, repeatedly multiplying by $A$ and normalizing converges to $\pm v_1$, and the tangent of the angle error shrinks by $|\lambda_2/\lambda_1|$ every step.

**Power iteration**

1. Start from a random unit vector $x$ (random ensures $c_1\ne0$).
2. Multiply: $w=Ax$.
3. Normalize: $x=w/\lVert w\rVert$.
4. Estimate $\hat\lambda=x^\top Ax$; repeat until $x$ stops turning.

## See it

> Figure: x → Ax: the symmetric preset [[2, 1], [1, 2]] stretches the ring into an ellipse; only the highlighted directions (0.71, 0.71) and (0.71, −0.71) stay on their dashed lines, scaled by λ₁ = 3 and λ₂ = 1 (trace 4, det 3). The rotation preset gives λ = 0.80 ± 0.60i and no real direction; shear has a single one. Power iteration on [[2, 0.5], [0.5, 1]] (λ₁ = 2.207, λ₂ = 0.793) starts 75.6° away and is within 3.72° after four steps — the tangent of the error falls by λ₂/λ₁ = 0.359 per step.

## Worked example

$A=\begin{pmatrix}2&1\\1&2\end{pmatrix}$: $\operatorname{tr}A=4$, $\det A=3$, so $\lambda^2-4\lambda+3=(\lambda-3)(\lambda-1)=0$ and $\lambda_1=3$, $\lambda_2=1$. For $\lambda_1=3$, $A-3I=\begin{pmatrix}-1&1\\1&-1\end{pmatrix}$ kills $v_1=(1,1)$; check $A(1,1)=(3,3)$. For $\lambda_2=1$, $v_2=(1,-1)$; check $A(1,-1)=(1,-1)$. The eigenvectors are orthogonal, as the spectral theorem promises.

Powers: $A^{10}=Q\operatorname{diag}(3^{10},1)Q^\top=\tfrac12\begin{pmatrix}59050&59048\\59048&59050\end{pmatrix}=\begin{pmatrix}29525&29524\\29524&29525\end{pmatrix}$ — no repeated multiplication needed.

Power iteration from $x_0=(1,0)$ (unnormalized): $(2,1)\to(5,4)\to(14,13)\to(41,40)$. The angle to $(1,1)$ is $18.43°\to6.34°\to2.12°\to0.71°$; its tangent goes $\tfrac13,\tfrac19,\tfrac1{27},\tfrac1{81}$, shrinking by exactly $\lambda_2/\lambda_1=1/3$ each step.

## In code

```python
import numpy as np

A = np.array([[2.0, 1.0], [1.0, 2.0]])
lam, V = np.linalg.eig(A)           # lam ≈ [3., 1.]; columns of V are unit eigenvectors
print(np.allclose(A @ V, V * lam))  # True: A v_i = λ_i v_i, column by column
print(lam.sum(), lam.prod())        # ≈ 4.0 3.0  = trace, determinant

w, Q = np.linalg.eigh(A)            # symmetric: w = [1., 3.] ascending, Q orthonormal
print(np.allclose(Q @ np.diag(w) @ Q.T, A))   # True: A = Q Λ Qᵀ

x = np.array([1.0, 0.0])            # power iteration
for _ in range(30):
    x = A @ x
    x /= np.linalg.norm(x)
print(x, x @ A @ x)                 # ≈ [0.7071 0.7071] 3.0
```

eig for general matrices, eigh for symmetric ones (real, sorted, orthonormal), and a hand-written power iteration.

- **PCA:** Principal components are the eigenvectors of the covariance matrix; each $\lambda_i$ is the variance captured along $q_i$.
- **PageRank & Markov chains:** The stationary distribution is the eigenvector of the transition matrix with $\lambda=1$, found by power iteration over the web graph.
- **Spectral clustering:** The eigenvectors of a graph Laplacian with the smallest eigenvalues embed nodes so that k-means can separate weakly connected groups.
- **Optimization curvature:** Hessian eigenvalues give curvature along each direction: gradient descent on a quadratic diverges unless the step size is below $2/\lambda_{\max}$, and mixed signs mark a saddle point.

> **Not every matrix is diagonalizable:** A real matrix can have complex eigenvalues (rotations) or too few eigenvectors (shears), and its eigenvectors need not be orthogonal. Only symmetric matrices guarantee a real, orthonormal eigenbasis — use `np.linalg.eigh` for them; `eig` on a nearly symmetric covariance can return tiny spurious imaginary parts.

> **Eigenvectors have no sign or scale:** If $v$ is an eigenvector, so is $-v$ and $5v$. Libraries return unit vectors with arbitrary sign, so PCA loadings can flip between runs or library versions. Compare subspaces or $|q_i^\top q_j|$, not raw coordinates.

**Question:** What are the eigenvalues of $A=\begin{pmatrix}4&1\\2&3\end{pmatrix}$?

- A. 4 and 3
- B. 5 and 2
- C. 7 and 10
- D. 6 and 1

**Answer:** B. 5 and 2 — $\operatorname{tr}A=7$, $\det A=12-2=10$, so $\lambda^2-7\lambda+10=(\lambda-5)(\lambda-2)=0$. Check: $5+2=7$ and $5\cdot2=10$.

**Question:** A symmetric matrix has eigenvalues $\lambda_1=2$ and $\lambda_2=1.8$ (all others smaller). By what factor does power iteration shrink the tangent of its angle error per step?

- A. 0.1
- B. 0.5
- C. 0.9
- D. 1.8

**Answer:** C. 0.9 — The $v_2$ component decays relative to $v_1$ by $\lambda_2/\lambda_1=1.8/2=0.9$ per step — slow: about 66 steps to gain three digits, since $0.9^{66}\approx10^{-3}$.

**Key points**

- $Av=\lambda v$: eigenvectors are the directions $A$ only scales; $\lambda$ is the scale.
- Eigenvalues solve $\det(A-\lambda I)=0$; $\sum\lambda_i=\operatorname{tr}A$ and $\prod\lambda_i=\det A$.
- Symmetric matrices factor as $Q\Lambda Q^\top$ with real $\lambda$ and orthonormal $Q$ — the basis of PCA.
- Power iteration finds the dominant eigenvector at rate $|\lambda_2/\lambda_1|$ per step.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
