Common Distributions

Bernoulli, binomial, Poisson, uniform, exponential and Gaussian each model one recurring kind of randomness, fixed by one or two parameters.

6 min read407 wordscore4 viewsupdated 2026-10-03

Definition

A probability distribution says how likely each value of a random variable is: a probability mass function (PMF) p(k)=P(X=k)p(k)=P(X=k) for a discrete variable, a probability density function (PDF) f(x)f(x) for a continuous one. Six families cover most of the randomness a data scientist meets, each fixed by one or two parameters.

Which one, when

Bernoulli (p)
One yes/no trial. P(X=1)=pP(X=1)=p. Mean pp, variance p(1−p)p(1-p). Click, churn, default.
Binomial (n, p)
Successes in nn independent Bernoulli(pp) trials. P(X=k)=(nk)pk(1−p)n−kP(X=k)=\binom{n}{k}p^k(1-p)^{n-k}. Mean npnp, variance np(1−p)np(1-p).
Poisson (λ)
Count of independent events in a fixed window at average rate λ\lambda. P(X=k)=e−λλk/k!P(X=k)=e^{-\lambda}\lambda^k/k!. Mean == variance =λ=\lambda.
Uniform (a, b)
Every value in [a,b][a,b] equally likely. f(x)=1/(b−a)f(x)=1/(b-a). Mean (a+b)/2(a+b)/2, variance (b−a)2/12(b-a)^2/12.
Exponential (λ)
Waiting time to the next event of a rate-λ\lambda process. f(x)=λe−λxf(x)=\lambda e^{-\lambda x} for x≥0x\ge0. Mean 1/λ1/\lambda, variance 1/λ21/\lambda^2.
Gaussian (μ, σ²)
Sum of many small independent effects. f(x)=1σ2πe−(x−μ)2/(2σ2)f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/(2\sigma^2)}. Mean μ\mu, variance σ2\sigma^2.

The intuition

Choose a distribution by asking what generated the data. A single yes/no → Bernoulli. The number of yeses in a fixed number of tries → binomial. A count with no fixed ceiling, of independent events at a steady rate → Poisson. The time until such an event → exponential. A measurement that is the sum of many small independent influences → Gaussian. No reason to prefer any value in a range → uniform.

The families are linked. A binomial is a sum of Bernoullis. A Poisson is the binomial's limit for many trials with a small success probability. Exponential gaps between events produce Poisson counts. And by the central limit theorem, a binomial with large np(1−p)np(1-p) is close to a Gaussian.

The math

∑kp(k)=1,∫−∞∞f(x) dx=1,F(x)=P(X≤x)\sum_k p(k)=1,\qquad \int_{-\infty}^{\infty}f(x)\,dx=1,\qquad F(x)=P(X\le x)
FF
cumulative distribution function (CDF); P(a<X≤b)=F(b)−F(a)P(a<X\le b)=F(b)-F(a)
f(x)f(x)
a density, not a probability — it can exceed 1; only areas ∫abf\int_a^b f are probabilities

Binomial moments by linearity. Write X=∑i=1nBiX=\sum_{i=1}^{n}B_i with independent Bi∼Bernoulli(p)B_i\sim\text{Bernoulli}(p). Then E[X]=np\mathbb{E}[X]=np and, since variances of independent terms add, Var⁡(X)=np(1−p)\operatorname{Var}(X)=np(1-p).

Poisson as a limit. Split a window into nn tiny slots, each holding an event with probability λ/n\lambda/n. As n→∞n\to\infty the binomial count converges:

(nk)(λn)k(1−λn)n−k ⟶ e−λλkk!\binom{n}{k}\Big(\frac{\lambda}{n}\Big)^{k}\Big(1-\frac{\lambda}{n}\Big)^{n-k}\ \longrightarrow\ \frac{e^{-\lambda}\lambda^{k}}{k!}

Memorylessness. The exponential is the only continuous distribution with P(X>s+t∣X>s)=P(X>t)P(X>s+t\mid X>s)=P(X>t): having already waited ss tells you nothing about the remaining wait.

See it

PMF P(X = k)0.110.21024681012141618

models: number of successes in n independent Bernoulli(p) trials

mean = np = 6.000variance = np(1−p) = 4.200
fig.Switch families and drag the parameters; mean and variance update next to their formulas. With Binomial n = 60, p = 0.05 the Poisson(3) overlay differs from the bars by at most 0.006.

Worked example

A site gets 2 sign-ups per hour on average, independently: Poisson with λ=2\lambda=2. P(none in an hour)=e−2≈0.135P(\text{none in an hour})=e^{-2}\approx0.135. P(at least 3)=1−e−2(1+2+222)=1−5e−2≈0.323P(\text{at least }3)=1-e^{-2}(1+2+\tfrac{2^2}{2})=1-5e^{-2}\approx0.323.

The wait for the next sign-up is Exponential(λ=2\lambda=2): mean 1/λ=301/\lambda=30 minutes, and P(wait>1 h)=e−2≈0.135P(\text{wait}>1\text{ h})=e^{-2}\approx0.135 — the same event as "no sign-ups in the next hour", seen from the other side.

In code

python
1from scipy import stats
2
3stats.poisson(mu=2).pmf(0) # 0.1353 P(no sign-ups in an hour)
41 - stats.poisson(mu=2).cdf(2) # 0.3233 P(at least 3)
5stats.expon(scale=1/2).sf(1) # 0.1353 P(wait > 1 h); scale = 1/λ
6stats.binom(n=60, p=0.05).pmf(3) # 0.2298 vs stats.poisson(3).pmf(3) = 0.2240
scipy.stats: every family has pmf/pdf, cdf, sf (= 1 − cdf), mean, var and rvs.

Where it's used

Generalized linear models

Logistic regression assumes a Bernoulli target, Poisson regression a Poisson count; the distribution fixes the loss (its negative log-likelihood).

A/B testing

Conversions out of n visitors are binomial; a test compares two binomial proportions.

Capacity planning

Poisson arrivals and exponential service times size servers, call centres and rate limits.

Initialization & priors

Network weights are drawn from uniform or Gaussian distributions with variance scaled to layer width (Xavier, He).

Check yourself

Defects occur independently at 0.5 per chip on average. What is the probability that a chip has none?

Check yourself

Which statement about a density f(x)f(x) is true?

Remember

  • ✓Match the process: trial → Bernoulli, successes in n trials → binomial, open-ended count → Poisson, waiting time → exponential, sum of many effects → Gaussian.
  • ✓Binomial = sum of Bernoullis; Poisson = its rare-event limit with np=λnp=\lambda.
  • ✓Poisson has mean = variance = λ — check this against your data before using it.
  • ✓Densities are not probabilities; only their integrals are.
← back to Mathematics roadmap