Conditional Probability & Independence

P(A|B) = P(A∩B)/P(B) restricts the world to B; A and B are independent when knowing B leaves P(A) unchanged.

5 min read451 wordscore3 viewsupdated 2026-10-03

Definition

The conditional probability of AA given BB, written P(A∣B)P(A\mid B), is the probability that AA occurs once we know BB has occurred — the share of BB's probability that also lies in AA. Events AA and BB are independent when P(A∣B)=P(A)P(A\mid B)=P(A): learning BB tells you nothing about AA.

At a glance

Notation
P(A∣B)P(A\mid B), "probability of AA given BB"; A∩BA\cap B = both occur
Defined when
P(B)>0P(B)>0
Product rule
P(A∩B)=P(A∣B) P(B)P(A\cap B)=P(A\mid B)\,P(B)
Independence
P(A∩B)=P(A) P(B)P(A\cap B)=P(A)\,P(B)
Prerequisites
events, sample space Ω\Omega, 0≤P≤10\le P\le1
Used in
Bayes' theorem, Naive Bayes, language models, A/B tests, every likelihood

The intuition

Picture probability as area in a square of total area 1 — the sample space Ω\Omega of everything that could happen. Learning that BB happened deletes everything outside BB. What is left is a smaller world, and probabilities must be measured relative to it: P(A∣B)P(A\mid B) is the fraction of BB that is also AA.

Independence means that zooming into BB changes nothing: AA covers the same fraction of BB as of the whole square. It is a property of the numbers, not of causal mechanisms — two causally unrelated events are statistically dependent whenever a third factor drives both.

The math

P(A∣B)=P(A∩B)P(B),P(B)>0P(A\mid B)=\frac{P(A\cap B)}{P(B)},\qquad P(B)>0
A∩BA\cap B
outcomes in both AA and BB
P(B)P(B)
the normalizer: B becomes the new total

Rearranged, this is the product rule. Applied repeatedly it factorizes any joint probability into a chain of conditionals:

P(A1∩⋯∩An)=P(A1) P(A2∣A1)⋯P(An∣A1,…,An−1)P(A_1\cap\cdots\cap A_n)=P(A_1)\,P(A_2\mid A_1)\cdots P(A_n\mid A_1,\dots,A_{n-1})

Law of total probability. If B1,…,BkB_1,\dots,B_k partition Ω\Omega — exactly one of them happens — then any probability is a weighted average of conditional ones:

P(A)=∑i=1kP(A∣Bi) P(Bi)P(A)=\sum_{i=1}^{k}P(A\mid B_i)\,P(B_i)

Independence. AA and BB are independent iff P(A∩B)=P(A)P(B)P(A\cap B)=P(A)P(B). The definition is symmetric, and when P(B)>0P(B)>0 it is equivalent to P(A∣B)=P(A)P(A\mid B)=P(A). Random variables XX and YY are independent when every pair of events {X∈S}\{X\in S\}, {Y∈T}\{Y\in T\} is. Conditional independence, P(A∩B∣C)=P(A∣C) P(B∣C)P(A\cap B\mid C)=P(A\mid C)\,P(B\mid C), is a different property: neither it nor plain independence implies the other.

See it

sample space Ω, area = 1A only0.180A∩B0.320B only0.080neither0.420A: width 0.50BP(A∩B)0.320P(A)·P(B)0.200A, B attractB's band has a step

P(A) = 0.500

P(B) = 0.400

P(A∩B) = 0.320

P(A|B) = P(A∩B)/P(B) = 0.800

P(B|A) = P(A∩B)/P(A) = 0.640

overlap slides P(A∩B) across its feasible range [0.00, 0.40]

fig.At the defaults P(A∩B) = 0.32 > P(A)P(B) = 0.20, so A and B attract: P(A|B) = 0.80 vs P(A) = 0.50. 'make independent' flattens B into a level band; 'condition on B' discards everything outside B and rescales it to area 1.

Worked example

Roll two fair dice. Let BB = "first die shows 3", so P(B)=16P(B)=\tfrac16.

AA = "sum is 8": P(A)=536≈0.139P(A)=\tfrac{5}{36}\approx0.139, but A∩B={(3,5)}A\cap B=\{(3,5)\}, so P(A∣B)=1/361/6=16≈0.167P(A\mid B)=\frac{1/36}{1/6}=\tfrac16\approx0.167. Knowing the first die changes the odds — dependent.

CC = "sum is 7": P(C)=636=16P(C)=\tfrac{6}{36}=\tfrac16, and C∩B={(3,4)}C\cap B=\{(3,4)\} gives P(C∣B)=16P(C\mid B)=\tfrac16 too. Whatever the first die shows, exactly one value of the second makes 7 — independent, even though the sum is computed from that die.

In code

python
1import numpy as np
2
3rng = np.random.default_rng(0)
4d1, d2 = rng.integers(1, 7, size=(2, 1_000_000))
5A, B, C = (d1 + d2 == 8), (d1 == 3), (d1 + d2 == 7)
6
7A[B].mean(), A.mean() # P(A|B) ≈ 0.167 vs P(A) ≈ 0.139 → dependent
8C[B].mean(), C.mean() # P(C|B) ≈ 0.167 vs P(C) ≈ 0.167 → independent
Conditioning is filtering: P(A|B) is the frequency of A among the rows where B holds.

Where it's used

Naive Bayes

Assumes features are conditionally independent given the class, P(x∣y)=∏jP(xj∣y)P(x\mid y)=\prod_j P(x_j\mid y) — the classic spam filter.

Language models

The chain rule P(x1,…,xn)=∏tP(xt∣x<t)P(x_1,\dots,x_n)=\prod_t P(x_t\mid x_{<t}) is exactly how an LLM scores a sentence.

Analytics & A/B tests

Conversion rate given a variant, churn given a segment: every dashboard filter computes a conditional probability.

Feature screening

If P(y∣xj)=P(y)P(y\mid x_j)=P(y) for every value of xjx_j, feature jj carries no information about the label.

Check yourself

P(A)=0.3P(A)=0.3, P(B)=0.5P(B)=0.5, P(A∩B)=0.15P(A\cap B)=0.15. Are AA and BB independent?

Check yourself

A fair coin is flipped twice. Given that at least one flip is heads, what is the probability that both are heads?

Remember

  • ✓P(A∣B)=P(A∩B)/P(B)P(A\mid B)=P(A\cap B)/P(B): restrict the world to BB and renormalize.
  • ✓The product rule and chain rule factorize any joint probability.
  • ✓Independent ⇔ P(A∩B)=P(A)P(B)P(A\cap B)=P(A)P(B) ⇔ P(A∣B)=P(A)P(A\mid B)=P(A).
  • ✓P(A∣B)≠P(B∣A)P(A\mid B)\ne P(B\mid A), and disjoint events are not independent.
← back to Mathematics roadmap