Eigenvectors & Eigenvalues

An eigenvector keeps its direction under A and is only scaled: Av = λv.

6 min read508 wordscore5 viewsupdated 2026-10-03

Definition

An eigenvector of a square matrix AA is a nonzero vector vv that AA maps onto its own line: Av=λvAv=\lambda v. The scalar λ\lambda is the corresponding eigenvalue — the factor by which that direction is stretched (negative λ\lambda flips it).

At a glance

Notation
Av=λvAv=\lambda v; A=VΛV−1A=V\Lambda V^{-1} with eigenvectors as columns of VV and Λ=diag⁡(λ1,…,λn)\Lambda=\operatorname{diag}(\lambda_1,\dots,\lambda_n)
Input → output
a square n×nn\times n matrix → nn eigenvalues (with multiplicity, possibly complex) and their eigenvectors
Key property
tr⁡A=∑iλi\operatorname{tr}A=\sum_i\lambda_i, det⁡A=∏iλi\det A=\prod_i\lambda_i; symmetric AA has real λ\lambda and orthonormal eigenvectors
Prerequisites
matrix multiplication, determinant, linear systems & rank
Used in
PCA, PageRank, spectral clustering, Markov chains, curvature of loss surfaces

The intuition

A matrix moves almost every vector off its own line: it turns the vector as well as stretching it. Eigenvectors are the exceptions — the directions the map leaves pointing the same way. Along them, a complicated matrix acts like a plain number.

That is why they matter. If you write any vector in eigenvector coordinates, applying AA just multiplies each coordinate by its own λ\lambda, and applying AA a hundred times multiplies by λ100\lambda^{100}. Directions with ∣λ∣>1|\lambda|>1 blow up, those with ∣λ∣<1|\lambda|<1 die out, and the largest ∣λ∣|\lambda| eventually dominates everything.

The math

Av=λvAv=\lambda v means (A−λI)v=0(A-\lambda I)v=0 has a nonzero solution, i.e. A−λIA-\lambda I is singular. So the eigenvalues are the roots of the characteristic polynomial, and each eigenvector spans part of the null space of A−λIA-\lambda I:

det⁡(A−λI)=0,(A−λI) v=0, v≠0\det(A-\lambda I)=0,\qquad (A-\lambda I)\,v=0,\ v\ne0
II
identity matrix of the same size as A
λ\lambda
an eigenvalue — a root of the degree-n polynomial

For a 2×22\times2 matrix the polynomial is λ2−(tr⁡A)λ+det⁡A\lambda^2-(\operatorname{tr}A)\lambda+\det A, so

λ1,2=tr⁡A2±(tr⁡A)24−det⁡A\lambda_{1,2}=\frac{\operatorname{tr}A}{2}\pm\sqrt{\frac{(\operatorname{tr}A)^2}{4}-\det A}
tr⁡A\operatorname{tr}A
trace, a11+a22a_{11}+a_{22}
det⁡A\det A
a11a22−a12a21a_{11}a_{22}-a_{12}a_{21}

A negative discriminant gives a complex pair: a rotation has no real direction it preserves. A zero discriminant gives a repeated eigenvalue, which may come with only one eigenvector — the shear (1101)\begin{pmatrix}1&1\\0&1\end{pmatrix} has λ=1\lambda=1 twice but only the direction (1,0)(1,0).

Diagonalization. If AA has nn independent eigenvectors, stack them as the columns of VV: then AV=VΛAV=V\Lambda, so A=VΛV−1A=V\Lambda V^{-1} and powers become cheap, Ak=VΛkV−1A^k=V\Lambda^kV^{-1}. For symmetric AA the spectral theorem guarantees real eigenvalues and an orthonormal V=QV=Q:

A=QΛQ⊤=∑i=1nλi qiqi⊤,λmax⁡=max⁡∥x∥=1x⊤AxA=Q\Lambda Q^\top=\sum_{i=1}^n\lambda_i\,q_iq_i^\top,\qquad \lambda_{\max}=\max_{\lVert x\rVert=1}x^\top Ax
qiq_i
orthonormal eigenvectors (columns of Q)
x⊤Axx^\top Ax
Rayleigh quotient of a unit vector x

Power iteration. Write x0=∑icivix_0=\sum_i c_iv_i. Then Akx0=∑iciλikvi=λ1k(c1v1+∑i>1ci(λi/λ1)kvi)A^kx_0=\sum_i c_i\lambda_i^kv_i=\lambda_1^k\big(c_1v_1+\sum_{i>1}c_i(\lambda_i/\lambda_1)^kv_i\big). If ∣λ1∣>∣λ2∣|\lambda_1|>|\lambda_2| and c1≠0c_1\ne0, repeatedly multiplying by AA and normalizing converges to ±v1\pm v_1, and the tangent of the angle error shrinks by ∣λ2/λ1∣|\lambda_2/\lambda_1| every step.

Power iteration

  1. 1Start from a random unit vector xx (random ensures c1≠0c_1\ne0).
  2. 2Multiply: w=Axw=Ax.
  3. 3Normalize: x=w/∥w∥x=w/\lVert w\rVert.
  4. 4Estimate λ^=x⊤Ax\hat\lambda=x^\top Ax; repeat until xx stops turning.

See it

eigen-directions stay on their own line
x → (1 − t)·x + t·Ax, t = 1.00

A =

t
λ₁ = 3.00 v₁ = (0.71, 0.71)λ₂ = 1.00 v₂ = (0.71, -0.71)trace = 4.00 = λ₁ + λ₂det = 3.00 = λ₁λ₂
fig.x → Ax: the symmetric preset [[2, 1], [1, 2]] stretches the ring into an ellipse; only the highlighted directions (0.71, 0.71) and (0.71, −0.71) stay on their dashed lines, scaled by λ₁ = 3 and λ₂ = 1 (trace 4, det 3). The rotation preset gives λ = 0.80 ± 0.60i and no real direction; shear has a single one. Power iteration on [[2, 0.5], [0.5, 1]] (λ₁ = 2.207, λ₂ = 0.793) starts 75.6° away and is within 3.72° after four steps — the tangent of the error falls by λ₂/λ₁ = 0.359 per step.

Worked example

A=(2112)A=\begin{pmatrix}2&1\\1&2\end{pmatrix}: tr⁡A=4\operatorname{tr}A=4, det⁡A=3\det A=3, so λ2−4λ+3=(λ−3)(λ−1)=0\lambda^2-4\lambda+3=(\lambda-3)(\lambda-1)=0 and λ1=3\lambda_1=3, λ2=1\lambda_2=1. For λ1=3\lambda_1=3, A−3I=(−111−1)A-3I=\begin{pmatrix}-1&1\\1&-1\end{pmatrix} kills v1=(1,1)v_1=(1,1); check A(1,1)=(3,3)A(1,1)=(3,3). For λ2=1\lambda_2=1, v2=(1,−1)v_2=(1,-1); check A(1,−1)=(1,−1)A(1,-1)=(1,-1). The eigenvectors are orthogonal, as the spectral theorem promises.

Powers: A10=Qdiag⁡(310,1)Q⊤=12(59050590485904859050)=(29525295242952429525)A^{10}=Q\operatorname{diag}(3^{10},1)Q^\top=\tfrac12\begin{pmatrix}59050&59048\\59048&59050\end{pmatrix}=\begin{pmatrix}29525&29524\\29524&29525\end{pmatrix} — no repeated multiplication needed.

Power iteration from x0=(1,0)x_0=(1,0) (unnormalized): (2,1)→(5,4)→(14,13)→(41,40)(2,1)\to(5,4)\to(14,13)\to(41,40). The angle to (1,1)(1,1) is 18.43°→6.34°→2.12°→0.71°18.43°\to6.34°\to2.12°\to0.71°; its tangent goes 13,19,127,181\tfrac13,\tfrac19,\tfrac1{27},\tfrac1{81}, shrinking by exactly λ2/λ1=1/3\lambda_2/\lambda_1=1/3 each step.

In code

python
1import numpy as np
2
3A = np.array([[2.0, 1.0], [1.0, 2.0]])
4lam, V = np.linalg.eig(A) # lam ≈ [3., 1.]; columns of V are unit eigenvectors
5print(np.allclose(A @ V, V * lam)) # True: A v_i = λ_i v_i, column by column
6print(lam.sum(), lam.prod()) # ≈ 4.0 3.0 = trace, determinant
7
8w, Q = np.linalg.eigh(A) # symmetric: w = [1., 3.] ascending, Q orthonormal
9print(np.allclose(Q @ np.diag(w) @ Q.T, A)) # True: A = Q Λ Qᵀ
10
11x = np.array([1.0, 0.0]) # power iteration
12for _ in range(30):
13 x = A @ x
14 x /= np.linalg.norm(x)
15print(x, x @ A @ x) # ≈ [0.7071 0.7071] 3.0
eig for general matrices, eigh for symmetric ones (real, sorted, orthonormal), and a hand-written power iteration.

Where it's used

PCA

Principal components are the eigenvectors of the covariance matrix; each λi\lambda_i is the variance captured along qiq_i.

PageRank & Markov chains

The stationary distribution is the eigenvector of the transition matrix with λ=1\lambda=1, found by power iteration over the web graph.

Spectral clustering

The eigenvectors of a graph Laplacian with the smallest eigenvalues embed nodes so that k-means can separate weakly connected groups.

Optimization curvature

Hessian eigenvalues give curvature along each direction: gradient descent on a quadratic diverges unless the step size is below 2/λmax⁡2/\lambda_{\max}, and mixed signs mark a saddle point.

Check yourself

What are the eigenvalues of A=(4123)A=\begin{pmatrix}4&1\\2&3\end{pmatrix}?

Check yourself

A symmetric matrix has eigenvalues λ1=2\lambda_1=2 and λ2=1.8\lambda_2=1.8 (all others smaller). By what factor does power iteration shrink the tangent of its angle error per step?

Remember

  • ✓Av=λvAv=\lambda v: eigenvectors are the directions AA only scales; λ\lambda is the scale.
  • ✓Eigenvalues solve det⁡(A−λI)=0\det(A-\lambda I)=0; ∑λi=tr⁡A\sum\lambda_i=\operatorname{tr}A and ∏λi=det⁡A\prod\lambda_i=\det A.
  • ✓Symmetric matrices factor as QΛQ⊤Q\Lambda Q^\top with real λ\lambda and orthonormal QQ — the basis of PCA.
  • ✓Power iteration finds the dominant eigenvector at rate ∣λ2/λ1∣|\lambda_2/\lambda_1| per step.
← back to Mathematics roadmap