Definition
A probability distribution says how likely each value of a random variable is: a probability mass function (PMF) for a discrete variable, a probability density function (PDF) for a continuous one. Six families cover most of the randomness a data scientist meets, each fixed by one or two parameters.
Which one, when
- Bernoulli (p)
- One yes/no trial. . Mean , variance . Click, churn, default.
- Binomial (n, p)
- Successes in independent Bernoulli() trials. . Mean , variance .
- Poisson (λ)
- Count of independent events in a fixed window at average rate . . Mean variance .
- Uniform (a, b)
- Every value in equally likely. . Mean , variance .
- Exponential (λ)
- Waiting time to the next event of a rate- process. for . Mean , variance .
- Gaussian (μ, σ²)
- Sum of many small independent effects. . Mean , variance .
The intuition
Choose a distribution by asking what generated the data. A single yes/no → Bernoulli. The number of yeses in a fixed number of tries → binomial. A count with no fixed ceiling, of independent events at a steady rate → Poisson. The time until such an event → exponential. A measurement that is the sum of many small independent influences → Gaussian. No reason to prefer any value in a range → uniform.
The families are linked. A binomial is a sum of Bernoullis. A Poisson is the binomial's limit for many trials with a small success probability. Exponential gaps between events produce Poisson counts. And by the central limit theorem, a binomial with large is close to a Gaussian.
The math
- cumulative distribution function (CDF);
- a density, not a probability — it can exceed 1; only areas are probabilities
Binomial moments by linearity. Write with independent . Then and, since variances of independent terms add, .
Poisson as a limit. Split a window into tiny slots, each holding an event with probability . As the binomial count converges:
Memorylessness. The exponential is the only continuous distribution with : having already waited tells you nothing about the remaining wait.
See it
models: number of successes in n independent Bernoulli(p) trials
Worked example
A site gets 2 sign-ups per hour on average, independently: Poisson with . . .
The wait for the next sign-up is Exponential(): mean minutes, and — the same event as "no sign-ups in the next hour", seen from the other side.
In code
1from scipy import stats23stats.poisson(mu=2).pmf(0) # 0.1353 P(no sign-ups in an hour)41 - stats.poisson(mu=2).cdf(2) # 0.3233 P(at least 3)5stats.expon(scale=1/2).sf(1) # 0.1353 P(wait > 1 h); scale = 1/λ6stats.binom(n=60, p=0.05).pmf(3) # 0.2298 vs stats.poisson(3).pmf(3) = 0.2240Where it's used
Generalized linear models
Logistic regression assumes a Bernoulli target, Poisson regression a Poisson count; the distribution fixes the loss (its negative log-likelihood).
A/B testing
Conversions out of n visitors are binomial; a test compares two binomial proportions.
Capacity planning
Poisson arrivals and exponential service times size servers, call centres and rate limits.
Initialization & priors
Network weights are drawn from uniform or Gaussian distributions with variance scaled to layer width (Xavier, He).
Check yourself
Defects occur independently at 0.5 per chip on average. What is the probability that a chip has none?
Check yourself
Which statement about a density is true?
Remember
- ✓Match the process: trial → Bernoulli, successes in n trials → binomial, open-ended count → Poisson, waiting time → exponential, sum of many effects → Gaussian.
- ✓Binomial = sum of Bernoullis; Poisson = its rare-event limit with .
- ✓Poisson has mean = variance = λ — check this against your data before using it.
- ✓Densities are not probabilities; only their integrals are.