# Conditional Probability & Independence

> P(A|B) = P(A∩B)/P(B) restricts the world to B; A and B are independent when knowing B leaves P(A) unchanged.

Source: https://www.mlfoundry.in/learn/conditional-probability-independence  
Section: Mathematics › Probability  
Level: core · 5 min read · updated 2026-10-03

**The conditional probability:** of $A$ given $B$, written $P(A\mid B)$, is the probability that $A$ occurs once we know $B$ has occurred — the share of $B$'s probability that also lies in $A$. Events $A$ and $B$ are **independent** when $P(A\mid B)=P(A)$: learning $B$ tells you nothing about $A$.

- **Notation:** $P(A\mid B)$, "probability of $A$ given $B$"; $A\cap B$ = both occur
- **Defined when:** $P(B)>0$
- **Product rule:** $P(A\cap B)=P(A\mid B)\,P(B)$
- **Independence:** $P(A\cap B)=P(A)\,P(B)$
- **Prerequisites:** events, sample space $\Omega$, $0\le P\le1$
- **Used in:** Bayes' theorem, Naive Bayes, language models, A/B tests, every likelihood

## The intuition

Picture probability as area in a square of total area 1 — the sample space $\Omega$ of everything that could happen. Learning that $B$ happened deletes everything outside $B$. What is left is a smaller world, and probabilities must be measured relative to it: $P(A\mid B)$ is the fraction of $B$ that is also $A$.

Independence means that zooming into $B$ changes nothing: $A$ covers the same fraction of $B$ as of the whole square. It is a property of the numbers, not of causal mechanisms — two causally unrelated events are statistically dependent whenever a third factor drives both.

## The math

$$
P(A\mid B)=\frac{P(A\cap B)}{P(B)},\qquad P(B)>0
$$

- $A\cap B$: outcomes in both $A$ and $B$
- $P(B)$: the normalizer: B becomes the new total

Rearranged, this is the **product rule**. Applied repeatedly it factorizes any joint probability into a chain of conditionals:

$$
P(A_1\cap\cdots\cap A_n)=P(A_1)\,P(A_2\mid A_1)\cdots P(A_n\mid A_1,\dots,A_{n-1})
$$

**Law of total probability.** If $B_1,\dots,B_k$ partition $\Omega$ — exactly one of them happens — then any probability is a weighted average of conditional ones:

$$
P(A)=\sum_{i=1}^{k}P(A\mid B_i)\,P(B_i)
$$

**Independence.** $A$ and $B$ are independent iff $P(A\cap B)=P(A)P(B)$. The definition is symmetric, and when $P(B)>0$ it is equivalent to $P(A\mid B)=P(A)$. Random variables $X$ and $Y$ are independent when every pair of events $\{X\in S\}$, $\{Y\in T\}$ is. **Conditional independence**, $P(A\cap B\mid C)=P(A\mid C)\,P(B\mid C)$, is a different property: neither it nor plain independence implies the other.

## See it

> Figure: At the defaults P(A∩B) = 0.32 > P(A)P(B) = 0.20, so A and B attract: P(A|B) = 0.80 vs P(A) = 0.50. 'make independent' flattens B into a level band; 'condition on B' discards everything outside B and rescales it to area 1.

## Worked example

Roll two fair dice. Let $B$ = "first die shows 3", so $P(B)=\tfrac16$.

$A$ = "sum is 8": $P(A)=\tfrac{5}{36}\approx0.139$, but $A\cap B=\{(3,5)\}$, so $P(A\mid B)=\frac{1/36}{1/6}=\tfrac16\approx0.167$. Knowing the first die changes the odds — **dependent**.

$C$ = "sum is 7": $P(C)=\tfrac{6}{36}=\tfrac16$, and $C\cap B=\{(3,4)\}$ gives $P(C\mid B)=\tfrac16$ too. Whatever the first die shows, exactly one value of the second makes 7 — **independent**, even though the sum is computed from that die.

## In code

```python
import numpy as np

rng = np.random.default_rng(0)
d1, d2 = rng.integers(1, 7, size=(2, 1_000_000))
A, B, C = (d1 + d2 == 8), (d1 == 3), (d1 + d2 == 7)

A[B].mean(), A.mean()     # P(A|B) ≈ 0.167 vs P(A) ≈ 0.139 → dependent
C[B].mean(), C.mean()     # P(C|B) ≈ 0.167 vs P(C) ≈ 0.167 → independent
```

Conditioning is filtering: P(A|B) is the frequency of A among the rows where B holds.

- **Naive Bayes:** Assumes features are conditionally independent given the class, $P(x\mid y)=\prod_j P(x_j\mid y)$ — the classic spam filter.
- **Language models:** The chain rule $P(x_1,\dots,x_n)=\prod_t P(x_t\mid x_{<t})$ is exactly how an LLM scores a sentence.
- **Analytics & A/B tests:** Conversion rate given a variant, churn given a segment: every dashboard filter computes a conditional probability.
- **Feature screening:** If $P(y\mid x_j)=P(y)$ for every value of $x_j$, feature $j$ carries no information about the label.

> **P(A|B) ≠ P(B|A):** *Confusion of the inverse.* $P(\text{positive}\mid\text{disease})=0.99$ does not mean $P(\text{disease}\mid\text{positive})=0.99$ — for a rare disease it can be below 10%. Turning one into the other needs Bayes' theorem and the base rate.

> **Disjoint is not independent:** Mutually exclusive events with positive probability are strongly *dependent*: if $A$ happens, $B$ cannot, so $P(B\mid A)=0\ne P(B)$.

> **Simpson's paradox:** A treatment can have the higher success rate within every subgroup and the lower rate overall, when the subgroups are sized differently across treatments. Which conditional probability answers your question depends on the causal structure, not on the arithmetic.

**Question:** $P(A)=0.3$, $P(B)=0.5$, $P(A\cap B)=0.15$. Are $A$ and $B$ independent?

- A. Yes
- B. No — they overlap
- C. No — P(A|B) = 0.5
- D. Cannot tell

**Answer:** A. Yes — $P(A)P(B)=0.3\cdot0.5=0.15=P(A\cap B)$. Equivalently $P(A\mid B)=0.15/0.5=0.3=P(A)$.

**Question:** A fair coin is flipped twice. Given that at least one flip is heads, what is the probability that both are heads?

- A. 1/2
- B. 1/3
- C. 1/4
- D. 2/3

**Answer:** B. 1/3 — $B=\{HH,HT,TH\}$ has probability $\tfrac34$ and $A\cap B=\{HH\}$ has $\tfrac14$, so $P(A\mid B)=\frac{1/4}{3/4}=\tfrac13$.

**Key points**

- $P(A\mid B)=P(A\cap B)/P(B)$: restrict the world to $B$ and renormalize.
- The product rule and chain rule factorize any joint probability.
- Independent ⇔ $P(A\cap B)=P(A)P(B)$ ⇔ $P(A\mid B)=P(A)$.
- $P(A\mid B)\ne P(B\mid A)$, and disjoint events are not independent.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
