Definition
A random variable assigns a number to each outcome of a random experiment. Its expectation is the probability-weighted average of its values; its variance is the expected squared distance from that average.
At a glance
- Notation
- variable, value; (PMF) or (PDF); ,
- Discrete
- Continuous
- Units
- and in the units of ; in squared units
- Prerequisites
- probability, sums, integrals
- Used in
- loss functions, SGD, risk and pricing, every estimator
The intuition
Place a weight equal to at each value on a beam. is where the beam balances — the centre of mass. Variance measures how far that mass sits from the balance point, squared; its square root is back in the units of and gives the typical size of a deviation.
The expectation need not be a possible value — a fair die has . It is a long-run average: by the law of large numbers, the mean of many independent draws approaches it.
The math
- , the second moment
Linearity holds for any random variables, dependent or not:
Proof (discrete): , because summing the joint distribution over one variable gives the other's marginal. Independence is never used.
Variance is not linear. Shifts drop out, scales come out squared, and sums pick up a covariance term:
For independent (or merely uncorrelated) variables the covariance is zero, so variances add — standard deviations do not. For a function , , but in general ; for convex , Jensen's inequality gives .
See it
Worked example
A fair die: and , so and .
The sum of two dice has (linearity) and, because the dice are independent, . Its standard deviation is — not .
In code
1import numpy as np23x = np.arange(1, 7)4p = np.full(6, 1 / 6) # fair die5mean = (x * p).sum() # 3.56var = ((x - mean) ** 2 * p).sum() # 2.9167 = 35/1278rng = np.random.default_rng(0)9rolls = rng.choice(x, p=p, size=(100_000, 2))10rolls.sum(1).mean(), rolls.sum(1).var() # ≈ 7.0, ≈ 5.83Where it's used
Loss functions
Training minimizes an expected loss ; the training-set average estimates it.
Stochastic gradient descent
A mini-batch gradient is unbiased — its expectation equals the full gradient — by linearity alone.
Decisions & pricing
Expected revenue, expected cost of an error, expected value of a bet: choose the action with the best expectation.
Estimator quality
Bias is and precision is ; error bars report .
Check yourself
and . What is ?
Check yourself
and are strongly dependent, with and . Then …
Remember
- ✓ — the balance point of the distribution.
- ✓Linearity: , always.
- ✓; .
- ✓Variances of independent variables add; in general.