Definition
The conditional probability of given , written , is the probability that occurs once we know has occurred — the share of 's probability that also lies in . Events and are independent when : learning tells you nothing about .
At a glance
- Notation
- , "probability of given "; = both occur
- Defined when
- Product rule
- Independence
- Prerequisites
- events, sample space ,
- Used in
- Bayes' theorem, Naive Bayes, language models, A/B tests, every likelihood
The intuition
Picture probability as area in a square of total area 1 — the sample space of everything that could happen. Learning that happened deletes everything outside . What is left is a smaller world, and probabilities must be measured relative to it: is the fraction of that is also .
Independence means that zooming into changes nothing: covers the same fraction of as of the whole square. It is a property of the numbers, not of causal mechanisms — two causally unrelated events are statistically dependent whenever a third factor drives both.
The math
- outcomes in both and
- the normalizer: B becomes the new total
Rearranged, this is the product rule. Applied repeatedly it factorizes any joint probability into a chain of conditionals:
Law of total probability. If partition — exactly one of them happens — then any probability is a weighted average of conditional ones:
Independence. and are independent iff . The definition is symmetric, and when it is equivalent to . Random variables and are independent when every pair of events , is. Conditional independence, , is a different property: neither it nor plain independence implies the other.
See it
P(A) = 0.500
P(B) = 0.400
P(A∩B) = 0.320
P(A|B) = P(A∩B)/P(B) = 0.800
P(B|A) = P(A∩B)/P(A) = 0.640
overlap slides P(A∩B) across its feasible range [0.00, 0.40]
Worked example
Roll two fair dice. Let = "first die shows 3", so .
= "sum is 8": , but , so . Knowing the first die changes the odds — dependent.
= "sum is 7": , and gives too. Whatever the first die shows, exactly one value of the second makes 7 — independent, even though the sum is computed from that die.
In code
1import numpy as np23rng = np.random.default_rng(0)4d1, d2 = rng.integers(1, 7, size=(2, 1_000_000))5A, B, C = (d1 + d2 == 8), (d1 == 3), (d1 + d2 == 7)67A[B].mean(), A.mean() # P(A|B) ≈ 0.167 vs P(A) ≈ 0.139 → dependent8C[B].mean(), C.mean() # P(C|B) ≈ 0.167 vs P(C) ≈ 0.167 → independentWhere it's used
Naive Bayes
Assumes features are conditionally independent given the class, — the classic spam filter.
Language models
The chain rule is exactly how an LLM scores a sentence.
Analytics & A/B tests
Conversion rate given a variant, churn given a segment: every dashboard filter computes a conditional probability.
Feature screening
If for every value of , feature carries no information about the label.
Check yourself
, , . Are and independent?
Check yourself
A fair coin is flipped twice. Given that at least one flip is heads, what is the probability that both are heads?
Remember
- ✓: restrict the world to and renormalize.
- ✓The product rule and chain rule factorize any joint probability.
- ✓Independent ⇔ ⇔ .
- ✓, and disjoint events are not independent.