Definition
The Gaussian (normal) distribution is the continuous distribution with density — a symmetric bell centred at the mean , whose width is set by the standard deviation .
At a glance
- Notation
- ; the standard normal is
- Parameters
- mean (location), variance (spread)
- Key property
- 68.27% / 95.45% / 99.73% of the mass lies within / /
- Closed under
- shifting, scaling, sums of independent Gaussians, conditioning, marginalizing
- Prerequisites
- random variables, expectation and variance
- Used in
- noise models, z-tests and confidence intervals, anomaly detection, VAEs, weight initialization
The intuition
Add up many small independent effects — measurement errors, the thousands of genes that influence height — and the total is approximately Gaussian; that is the central limit theorem. The density falls off as in the number of standard deviations from the mean, so extremes become rare very fast: a deviation happens 0.27% of the time, a one about once in 1.7 million.
The math
- mean — centre of the bell
- standard deviation — distance from to each inflection point
Every Gaussian is a shifted, stretched standard normal. If , its z-score is , so every probability reduces to the standard normal CDF :
has no closed form; libraries evaluate it numerically. Independent Gaussians add: if and are independent, then — variances add, standard deviations do not.
In dimensions, with mean vector and covariance matrix :
- squared Mahalanobis distance — distance in standard deviations
Contours of equal density are ellipses (ellipsoids) whose axes are the eigenvectors of , with half-length per standard deviation. Among all distributions with a given mean and variance, the Gaussian has maximum entropy: it assumes nothing beyond those two numbers.
See it
2-D: Σ = [[1, ρ], [ρ, 1]], 300 samples
inside 1σ ellipse: 37.7% (theory 39.3%)
inside 2σ ellipse: 86.0% (theory 86.5%)
In d dimensions “±kσ” holds far less than in 1-D: P = 1 − e^(−k²/2) for d = 2.
Worked example
IQ scores are scaled to , . For : , so — about 2.3% of people. Between 85 and 115 (): . The 97.5th percentile is .
In code
1import numpy as np2from scipy.stats import norm34X = norm(loc=100, scale=15)5X.sf(130) # P(X > 130) = 0.02286X.cdf(115) - X.cdf(85) # 0.68277X.ppf(0.975) # 129.4 — the 97.5th percentile89rng = np.random.default_rng(0)10samples = rng.normal(100, 15, size=10_000)Where it's used
Noise models
Least squares is maximum likelihood under Gaussian noise; Kalman filters propagate Gaussian beliefs.
Inference
z-tests, standard errors and confidence intervals use the Gaussian approximation from the CLT.
Anomaly detection
Flag points with , or with low density under a fitted multivariate Gaussian.
Deep learning
He / Xavier initialization, the VAE prior , and the noise in diffusion models.
Check yourself
. What is ?
Check yourself
and are independent, each . The standard deviation of is…
Remember
- ✓ is fixed by its mean and variance.
- ✓Standardize with ; probabilities come from .
- ✓68.27% / 95.45% / 99.73% lie within 1 / 2 / 3σ — in 1-D only.
- ✓Variances of independent Gaussians add. Check real tails before assuming normality.