# Common Distributions

> Bernoulli, binomial, Poisson, uniform, exponential and Gaussian each model one recurring kind of randomness, fixed by one or two parameters.

Source: https://www.mlfoundry.in/learn/common-distributions  
Section: Mathematics › Probability  
Level: core · 6 min read · updated 2026-10-03

**A probability distribution:** says how likely each value of a random variable is: a **probability mass function** (PMF) $p(k)=P(X=k)$ for a discrete variable, a **probability density function** (PDF) $f(x)$ for a continuous one. Six families cover most of the randomness a data scientist meets, each fixed by one or two parameters.

**Which one, when**

- **Bernoulli (p):** One yes/no trial. $P(X=1)=p$. Mean $p$, variance $p(1-p)$. *Click, churn, default.*
- **Binomial (n, p):** Successes in $n$ independent Bernoulli($p$) trials. $P(X=k)=\binom{n}{k}p^k(1-p)^{n-k}$. Mean $np$, variance $np(1-p)$.
- **Poisson (λ):** Count of independent events in a fixed window at average rate $\lambda$. $P(X=k)=e^{-\lambda}\lambda^k/k!$. Mean $=$ variance $=\lambda$.
- **Uniform (a, b):** Every value in $[a,b]$ equally likely. $f(x)=1/(b-a)$. Mean $(a+b)/2$, variance $(b-a)^2/12$.
- **Exponential (λ):** Waiting time to the next event of a rate-$\lambda$ process. $f(x)=\lambda e^{-\lambda x}$ for $x\ge0$. Mean $1/\lambda$, variance $1/\lambda^2$.
- **Gaussian (μ, σ²):** Sum of many small independent effects. $f(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/(2\sigma^2)}$. Mean $\mu$, variance $\sigma^2$.

## The intuition

Choose a distribution by asking what generated the data. A single yes/no → **Bernoulli**. The number of yeses in a fixed number of tries → **binomial**. A count with no fixed ceiling, of independent events at a steady rate → **Poisson**. The time until such an event → **exponential**. A measurement that is the sum of many small independent influences → **Gaussian**. No reason to prefer any value in a range → **uniform**.

The families are linked. A binomial is a sum of Bernoullis. A Poisson is the binomial's limit for many trials with a small success probability. Exponential gaps between events produce Poisson counts. And by the central limit theorem, a binomial with large $np(1-p)$ is close to a Gaussian.

## The math

$$
\sum_k p(k)=1,\qquad \int_{-\infty}^{\infty}f(x)\,dx=1,\qquad F(x)=P(X\le x)
$$

- $F$: cumulative distribution function (CDF); $P(a<X\le b)=F(b)-F(a)$
- $f(x)$: a density, not a probability — it can exceed 1; only areas $\int_a^b f$ are probabilities

**Binomial moments by linearity.** Write $X=\sum_{i=1}^{n}B_i$ with independent $B_i\sim\text{Bernoulli}(p)$. Then $\mathbb{E}[X]=np$ and, since variances of independent terms add, $\operatorname{Var}(X)=np(1-p)$.

**Poisson as a limit.** Split a window into $n$ tiny slots, each holding an event with probability $\lambda/n$. As $n\to\infty$ the binomial count converges:

$$
\binom{n}{k}\Big(\frac{\lambda}{n}\Big)^{k}\Big(1-\frac{\lambda}{n}\Big)^{n-k}\ \longrightarrow\ \frac{e^{-\lambda}\lambda^{k}}{k!}
$$

**Memorylessness.** The exponential is the only continuous distribution with $P(X>s+t\mid X>s)=P(X>t)$: having already waited $s$ tells you nothing about the remaining wait.

## See it

> Figure: Switch families and drag the parameters; mean and variance update next to their formulas. With Binomial n = 60, p = 0.05 the Poisson(3) overlay differs from the bars by at most 0.006.

## Worked example

A site gets 2 sign-ups per hour on average, independently: Poisson with $\lambda=2$. $P(\text{none in an hour})=e^{-2}\approx0.135$. $P(\text{at least }3)=1-e^{-2}(1+2+\tfrac{2^2}{2})=1-5e^{-2}\approx0.323$.

The wait for the next sign-up is Exponential($\lambda=2$): mean $1/\lambda=30$ minutes, and $P(\text{wait}>1\text{ h})=e^{-2}\approx0.135$ — the same event as "no sign-ups in the next hour", seen from the other side.

## In code

```python
from scipy import stats

stats.poisson(mu=2).pmf(0)          # 0.1353  P(no sign-ups in an hour)
1 - stats.poisson(mu=2).cdf(2)      # 0.3233  P(at least 3)
stats.expon(scale=1/2).sf(1)        # 0.1353  P(wait > 1 h); scale = 1/λ
stats.binom(n=60, p=0.05).pmf(3)    # 0.2298  vs stats.poisson(3).pmf(3) = 0.2240
```

scipy.stats: every family has pmf/pdf, cdf, sf (= 1 − cdf), mean, var and rvs.

- **Generalized linear models:** Logistic regression assumes a Bernoulli target, Poisson regression a Poisson count; the distribution fixes the loss (its negative log-likelihood).
- **A/B testing:** Conversions out of n visitors are binomial; a test compares two binomial proportions.
- **Capacity planning:** Poisson arrivals and exponential service times size servers, call centres and rate limits.
- **Initialization & priors:** Network weights are drawn from uniform or Gaussian distributions with variance scaled to layer width (Xavier, He).

> **Check the parameterization:** SciPy's `expon(scale=...)` takes the mean $1/\lambda$, not the rate $\lambda$; NumPy's `exponential(scale)` does the same. Passing the rate gives a silently wrong distribution.

> **Overdispersed counts:** Real counts are often far more variable than their mean (bursty traffic, clustered defects). A Poisson model then understates uncertainty — use a negative binomial, which adds a dispersion parameter.

**Question:** Defects occur independently at 0.5 per chip on average. What is the probability that a chip has none?

- A. 0.5
- B. e^(−0.5) ≈ 0.61
- C. 0.39
- D. 0.95

**Answer:** B. e^(−0.5) ≈ 0.61 — Poisson with $\lambda=0.5$: $P(X=0)=e^{-0.5}\lambda^0/0!=e^{-0.5}\approx0.607$.

**Question:** Which statement about a density $f(x)$ is true?

- A. $f(x)\le1$ for all $x$
- B. $f(x)$ is the probability that $X=x$
- C. $P(a<X\le b)=\int_a^b f(x)\,dx$
- D. $f$ must be symmetric

**Answer:** C. $P(a<X\le b)=\int_a^b f(x)\,dx$ — Uniform$(0, 0.5)$ has $f=2$ on its support, and for a continuous variable $P(X=x)=0$ for every single $x$. Only integrals of $f$ are probabilities.

**Key points**

- Match the process: trial → Bernoulli, successes in n trials → binomial, open-ended count → Poisson, waiting time → exponential, sum of many effects → Gaussian.
- Binomial = sum of Bernoullis; Poisson = its rare-event limit with $np=\lambda$.
- Poisson has mean = variance = λ — check this against your data before using it.
- Densities are not probabilities; only their integrals are.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
