Definition
Bayes' theorem converts — how likely the evidence is if a hypothesis holds — into — how likely the hypothesis is given the evidence — by weighting with the prior : posterior ∝ likelihood × prior.
At a glance
- Statement
- Ingredients
- prior , likelihood , evidence
- Follows from
- the definition of conditional probability, applied in both directions
- Sequential use
- today's posterior is tomorrow's prior
- Prerequisites
- conditional probability, the law of total probability
- Used in
- diagnostics, spam filters, Bayesian A/B tests, MAP estimation
The intuition
Evidence doesn't speak for itself; it multiplies belief by a factor. A positive test is strong evidence only if it is much more likely under sick than under healthy — and even strong evidence cannot overcome a tiny prior. Bayes makes this exact: multiply what you believed by how well each hypothesis predicted what you saw, then renormalize so the beliefs sum to 1.
The math
Write both ways — — divide by , and expand by the law of total probability:
- prior — belief before the evidence
- likelihood — how well H predicts E
- posterior — belief after the evidence
In odds form the normalizer cancels: posterior odds equal the likelihood ratio times the prior odds.
For parameters, replace by and by data : . For a coin with heads-probability , a prior and heads in flips give a Beta posterior — the prior acts like heads and tails already seen:
See it
■ sick, test + 4
■ healthy, test + 36
■ sick, test − 0
P(sick | +) =
9.2%
grid: 4 of 40 positives are sick
Worked example
A disease affects 1% of people. A test detects it 90% of the time and false-alarms 9% of the time. After a positive test: . In odds form: prior odds , likelihood ratio , posterior odds , so .
In code
1from scipy.stats import beta23def posterior(prior, sens, fpr):4 evidence = sens * prior + fpr * (1 - prior)5 return sens * prior / evidence67posterior(0.01, 0.90, 0.09) # 0.091789# uniform Beta(1, 1) prior, 7 heads in 10 flips → Beta(8, 4)10post = beta(1 + 7, 1 + 3)11post.mean(), post.interval(0.95) # 0.667, (0.390, 0.891)Where it's used
Medical diagnostics
Post-test probability from prevalence, sensitivity and specificity.
Spam filtering
Naive Bayes multiplies per-word likelihood ratios into posterior odds.
Bayesian A/B tests
Report directly from Beta posteriors.
Regularization
MAP estimation with a Gaussian prior on weights is L2 regularization; a Laplace prior gives L1.
Check yourself
Same test (90% sensitivity, 9% false-positive rate), but prevalence is 50%. What is P(D | +)?
Check yourself
A prior on a coin's bias, then 3 heads and 1 tail. The posterior mean is…
Remember
- ✓Posterior ∝ likelihood × prior; divide by the evidence to normalize.
- ✓Odds form: posterior odds = likelihood ratio × prior odds.
- ✓Rare hypotheses need very strong evidence; P(E|H) is not P(H|E).
- ✓Beta prior + coin flips → Beta posterior; priors on weights ≡ regularization.