Covariance & Correlation

Covariance measures how two variables move together; correlation rescales it to [−1, 1] and captures only linear association.

5 min read388 wordscore3 viewsupdated 2026-10-03

Definition

Covariance Cov⁡(X,Y)=E[(X−μX)(Y−μY)]\operatorname{Cov}(X,Y)=\mathbb{E}\big[(X-\mu_X)(Y-\mu_Y)\big] measures whether two variables tend to sit above their means together (positive) or on opposite sides (negative). The Pearson correlation ρ=Cov⁡(X,Y)/(σXσY)\rho=\operatorname{Cov}(X,Y)/(\sigma_X\sigma_Y) rescales it to [−1,1][-1,1], free of units.

At a glance

Notation
Cov⁡(X,Y)\operatorname{Cov}(X,Y) or σXY\sigma_{XY}; ρXY\rho_{XY} (population), rr (sample)
Range
covariance: any real number, in units of XX × units of YY; correlation: [−1,1][-1,1], unitless
ρ = ±1
exactly when Y=aX+bY=aX+b with a≠0a\ne0; the sign of aa gives the sign of ρ\rho
ρ = 0
no linear association — dependence can remain
Covariance matrix
Σ=1n−1Xc⊤Xc\Sigma=\frac{1}{n-1}X_c^\top X_c for centred data XcX_c; symmetric, positive semi-definite
Used in
PCA, Gaussian models, portfolio risk, feature screening

The intuition

Centre both variables at their means. Every point now lies in one of four quadrants. Points top-right or bottom-left — xx and yy deviating the same way — contribute a positive product (x−xˉ)(y−yˉ)(x-\bar x)(y-\bar y); points in the other two quadrants contribute negative products. Covariance is the average of these signed rectangle areas: which way the cloud tilts, weighted by how far points stray.

Covariance depends on units — measure height in centimetres instead of metres and it grows 100×. Dividing by both standard deviations removes the units and leaves a pure number, the correlation, which is simply the covariance of the standardized variables zx=(x−xˉ)/sxz_x=(x-\bar x)/s_x and zy=(y−yˉ)/syz_y=(y-\bar y)/s_y.

The math

Cov⁡(X,Y)=E[XY]−μXμY,r=∑i(xi−xˉ)(yi−yˉ)∑i(xi−xˉ)2 ∑i(yi−yˉ)2\operatorname{Cov}(X,Y)=\mathbb{E}[XY]-\mu_X\mu_Y,\qquad r=\frac{\sum_i(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_i(x_i-\bar x)^2\,\sum_i(y_i-\bar y)^2}}
xˉ, yˉ\bar x,\ \bar y
sample means
rr
sample (Pearson) correlation

Cov⁡(X,X)=Var⁡(X)\operatorname{Cov}(X,X)=\operatorname{Var}(X), and covariance is bilinear: Cov⁡(aX+b, cY+d)=acCov⁡(X,Y)\operatorname{Cov}(aX+b,\,cY+d)=ac\operatorname{Cov}(X,Y). So correlation is unchanged by shifting either variable or rescaling it by a positive factor. By the Cauchy–Schwarz inequality ∣Cov⁡(X,Y)∣≤σXσY|\operatorname{Cov}(X,Y)|\le\sigma_X\sigma_Y, which is why ∣ρ∣≤1|\rho|\le1.

Correlation is regression in standardized units: the least-squares line of yy on xx has slope b=r sy/sxb=r\,s_y/s_x, and r2r^2 is the fraction of the variance of yy explained by that line. For a random vector, all pairwise covariances form the covariance matrix Σ\Sigma, and the variance of any linear combination follows from it:

Var⁡(w⊤X)=w⊤Σ w\operatorname{Var}(w^\top X)=w^\top\Sigma\,w

See it

x̄ȳ+ quadrants− quadrants

cov(X, Y) = 0.600

r = cov / (σₓσᵧ) = 0.600

Σ deviation products

+53.0−5.6

fig.Points in the green quadrants add to the covariance, red ones subtract. Set ρ and the sample r matches it exactly. Under 'where r misleads', a parabola and a circle give r ≈ 0 despite perfect dependence, and one outlier turns r = −0.88 into +0.67.

Worked example

Hours studied x=(1,2,3,4,5)x=(1,2,3,4,5), exam score y=(52,55,61,64,68)y=(52,55,61,64,68); means 33 and 6060. Deviations x−xˉ=(−2,−1,0,1,2)x-\bar x=(-2,-1,0,1,2) and y−yˉ=(−8,−5,1,4,8)y-\bar y=(-8,-5,1,4,8). Products sum to 16+5+0+4+16=4116+5+0+4+16=41, so sxy=41/4=10.25s_{xy}=41/4=10.25.

With sx=10/4=1.581s_x=\sqrt{10/4}=1.581 and sy=170/4=6.519s_y=\sqrt{170/4}=6.519: r=10.251.581⋅6.519=0.994r=\frac{10.25}{1.581\cdot6.519}=0.994. The fitted slope is r sy/sx=41/10=4.1r\,s_y/s_x=41/10=4.1 points per hour.

In code

python
1import numpy as np
2from scipy import stats
3
4x = np.array([1, 2, 3, 4, 5])
5y = np.array([52, 55, 61, 64, 68])
6np.cov(x, y) # [[2.5, 10.25], [10.25, 42.5]]
7np.corrcoef(x, y)[0, 1] # 0.9944
8stats.spearmanr(x, y) # statistic = 1.0: y rises monotonically with x
np.cov uses the n − 1 denominator by default; Spearman correlates ranks.

Where it's used

Feature screening

A correlation heatmap flags redundant features (|r| > 0.9) that destabilize linear-model weights.

PCA

Principal components are the eigenvectors of the covariance matrix.

Finance

Portfolio variance is w⊤Σww^\top\Sigma w; diversification works because correlations are below 1.

Gaussian models

A multivariate Gaussian is defined by a mean vector and a covariance matrix; GMMs and Kalman filters estimate Σ.

Check yourself

Temperatures are converted from °C to °F (F = 1.8·C + 32). The correlation between temperature and ice-cream sales…

Check yourself

X∼Uniform(−1,1)X\sim\text{Uniform}(-1,1) and Y=X2Y=X^2. The correlation between XX and YY is…

Remember

  • ✓Cov⁡(X,Y)=E[(X−μX)(Y−μY)]\operatorname{Cov}(X,Y)=\mathbb{E}[(X-\mu_X)(Y-\mu_Y)]: the average signed product of deviations.
  • ✓ρ=Cov⁡/(σXσY)∈[−1,1]\rho=\operatorname{Cov}/(\sigma_X\sigma_Y)\in[-1,1], unit-free; ±1\pm1 only for exact linear relations.
  • ✓r=0r=0 means no linear association — not independence.
  • ✓Correlation is not causation; plot the data, check outliers, consider Spearman.
← back to Mathematics roadmap