# The Gaussian Distribution

> Fully specified by mean and variance; about 68%, 95% and 99.7% of its mass lies within 1, 2 and 3 standard deviations.

Source: https://www.mlfoundry.in/learn/gaussian-distribution  
Section: Mathematics › Probability  
Level: core · 5 min read · updated 2026-10-03

**The Gaussian (normal) distribution:** $\mathcal N(\mu,\sigma^2)$ is the continuous distribution with density $f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}$ — a symmetric bell centred at the mean $\mu$, whose width is set by the standard deviation $\sigma$.

- **Notation:** $X\sim\mathcal N(\mu,\sigma^2)$; the standard normal is $Z\sim\mathcal N(0,1)$
- **Parameters:** mean $\mu\in\mathbb R$ (location), variance $\sigma^2>0$ (spread)
- **Key property:** 68.27% / 95.45% / 99.73% of the mass lies within $\pm1$ / $\pm2$ / $\pm3\sigma$
- **Closed under:** shifting, scaling, sums of independent Gaussians, conditioning, marginalizing
- **Prerequisites:** random variables, expectation and variance
- **Used in:** noise models, z-tests and confidence intervals, anomaly detection, VAEs, weight initialization

## The intuition

Add up many small independent effects — measurement errors, the thousands of genes that influence height — and the total is approximately Gaussian; that is the central limit theorem. The density falls off as $e^{-z^2/2}$ in the number of standard deviations $z$ from the mean, so extremes become rare very fast: a $3\sigma$ deviation happens 0.27% of the time, a $5\sigma$ one about once in 1.7 million.

## The math

$$
f(x)=\frac{1}{\sigma\sqrt{2\pi}}\exp\!\Big(-\frac{(x-\mu)^2}{2\sigma^2}\Big),\qquad \mathbb E[X]=\mu,\quad \operatorname{Var}(X)=\sigma^2
$$

- $\mu$: mean — centre of the bell
- $\sigma$: standard deviation — distance from $\mu$ to each inflection point

Every Gaussian is a shifted, stretched standard normal. If $X\sim\mathcal N(\mu,\sigma^2)$, its **z-score** $Z=(X-\mu)/\sigma$ is $\mathcal N(0,1)$, so every probability reduces to the standard normal CDF $\Phi$:

$$
P(X\le x)=\Phi\Big(\frac{x-\mu}{\sigma}\Big),\qquad \Phi(z)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-t^2/2}\,dt=\tfrac12\Big[1+\operatorname{erf}\big(z/\sqrt2\big)\Big]
$$

$\Phi$ has no closed form; libraries evaluate it numerically. **Independent Gaussians add**: if $X\sim\mathcal N(\mu_1,\sigma_1^2)$ and $Y\sim\mathcal N(\mu_2,\sigma_2^2)$ are independent, then $aX+bY\sim\mathcal N(a\mu_1+b\mu_2,\ a^2\sigma_1^2+b^2\sigma_2^2)$ — variances add, standard deviations do not.

In $d$ dimensions, with mean vector $\mu$ and covariance matrix $\Sigma$:

$$
\mathcal N(x\mid\mu,\Sigma)=\frac{1}{(2\pi)^{d/2}|\Sigma|^{1/2}}\exp\!\Big(-\tfrac12(x-\mu)^\top\Sigma^{-1}(x-\mu)\Big)
$$

- $(x-\mu)^\top\Sigma^{-1}(x-\mu)$: squared Mahalanobis distance — distance in standard deviations

Contours of equal density are ellipses (ellipsoids) whose axes are the eigenvectors of $\Sigma$, with half-length $\sqrt{\lambda_i}$ per standard deviation. Among all distributions with a given mean and variance, the Gaussian has maximum entropy: it assumes nothing beyond those two numbers.

## See it

> Figure: Drag x to read its z-score and tail area; at z = 1.5 the tail is 0.0668. Below: in 2-D the 1σ ellipse holds only 39.3% of the mass and the 2σ ellipse 86.5%.

## Worked example

IQ scores are scaled to $\mu=100$, $\sigma=15$. For $x=130$: $z=(130-100)/15=2$, so $P(X>130)=1-\Phi(2)=0.0228$ — about 2.3% of people. Between 85 and 115 ($\pm1\sigma$): $\Phi(1)-\Phi(-1)=0.6827$. The 97.5th percentile is $100+1.96\cdot15=129.4$.

## In code

```python
import numpy as np
from scipy.stats import norm

X = norm(loc=100, scale=15)
X.sf(130)                    # P(X > 130) = 0.0228
X.cdf(115) - X.cdf(85)       # 0.6827
X.ppf(0.975)                 # 129.4 — the 97.5th percentile

rng = np.random.default_rng(0)
samples = rng.normal(100, 15, size=10_000)
```

scipy.stats.norm: probabilities, quantiles and sampling.

- **Noise models:** Least squares is maximum likelihood under Gaussian noise; Kalman filters propagate Gaussian beliefs.
- **Inference:** z-tests, standard errors and confidence intervals use the Gaussian approximation from the CLT.
- **Anomaly detection:** Flag points with $|z|>3$, or with low density under a fitted multivariate Gaussian.
- **Deep learning:** He / Xavier initialization, the VAE prior $\mathcal N(0,I)$, and the noise in diffusion models.

> **Real data has heavier tails:** Incomes, file sizes and asset returns are not Gaussian. Under a Gaussian, a $5\sigma$ daily market move should happen about once in 7,000 years of trading; real markets have produced many within living memory. Check tails with a Q–Q plot before assuming normality.

> **A quick sanity check:** If clearly more than ~5% of supposedly normal data lies beyond $\pm2\sigma$, or more than ~0.3% beyond $\pm3\sigma$, the tails are heavier than Gaussian.

**Question:** $X\sim\mathcal N(50,\,10^2)$. What is $P(X<30)$?

- A. ≈ 0.3%
- B. ≈ 2.3%
- C. ≈ 5%
- D. ≈ 16%

**Answer:** B. ≈ 2.3% — $z=(30-50)/10=-2$, and $\Phi(-2)=0.0228$.

**Question:** $X$ and $Y$ are independent, each $\mathcal N(0,1)$. The standard deviation of $X+Y$ is…

- A. 2
- B. √2
- C. 1
- D. 0

**Answer:** B. √2 — Variances add: $\operatorname{Var}(X+Y)=1+1=2$, so the standard deviation is $\sqrt2\approx1.41$, not 2.

**Key points**

- $\mathcal N(\mu,\sigma^2)$ is fixed by its mean and variance.
- Standardize with $z=(x-\mu)/\sigma$; probabilities come from $\Phi$.
- 68.27% / 95.45% / 99.73% lie within 1 / 2 / 3σ — in 1-D only.
- Variances of independent Gaussians add. Check real tails before assuming normality.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
