Definition
Covariance measures whether two variables tend to sit above their means together (positive) or on opposite sides (negative). The Pearson correlation rescales it to , free of units.
At a glance
- Notation
- or ; (population), (sample)
- Range
- covariance: any real number, in units of × units of ; correlation: , unitless
- ρ = ±1
- exactly when with ; the sign of gives the sign of
- ρ = 0
- no linear association — dependence can remain
- Covariance matrix
- for centred data ; symmetric, positive semi-definite
- Used in
- PCA, Gaussian models, portfolio risk, feature screening
The intuition
Centre both variables at their means. Every point now lies in one of four quadrants. Points top-right or bottom-left — and deviating the same way — contribute a positive product ; points in the other two quadrants contribute negative products. Covariance is the average of these signed rectangle areas: which way the cloud tilts, weighted by how far points stray.
Covariance depends on units — measure height in centimetres instead of metres and it grows 100×. Dividing by both standard deviations removes the units and leaves a pure number, the correlation, which is simply the covariance of the standardized variables and .
The math
- sample means
- sample (Pearson) correlation
, and covariance is bilinear: . So correlation is unchanged by shifting either variable or rescaling it by a positive factor. By the Cauchy–Schwarz inequality , which is why .
Correlation is regression in standardized units: the least-squares line of on has slope , and is the fraction of the variance of explained by that line. For a random vector, all pairwise covariances form the covariance matrix , and the variance of any linear combination follows from it:
See it
cov(X, Y) = 0.600
r = cov / (σₓσᵧ) = 0.600
Σ deviation products
+53.0−5.6
Worked example
Hours studied , exam score ; means and . Deviations and . Products sum to , so .
With and : . The fitted slope is points per hour.
In code
1import numpy as np2from scipy import stats34x = np.array([1, 2, 3, 4, 5])5y = np.array([52, 55, 61, 64, 68])6np.cov(x, y) # [[2.5, 10.25], [10.25, 42.5]]7np.corrcoef(x, y)[0, 1] # 0.99448stats.spearmanr(x, y) # statistic = 1.0: y rises monotonically with xWhere it's used
Feature screening
A correlation heatmap flags redundant features (|r| > 0.9) that destabilize linear-model weights.
PCA
Principal components are the eigenvectors of the covariance matrix.
Finance
Portfolio variance is ; diversification works because correlations are below 1.
Gaussian models
A multivariate Gaussian is defined by a mean vector and a covariance matrix; GMMs and Kalman filters estimate Σ.
Check yourself
Temperatures are converted from °C to °F (F = 1.8·C + 32). The correlation between temperature and ice-cream sales…
Check yourself
and . The correlation between and is…
Remember
- ✓: the average signed product of deviations.
- ✓, unit-free; only for exact linear relations.
- ✓ means no linear association — not independence.
- ✓Correlation is not causation; plot the data, check outliers, consider Spearman.