Section 3 · Lesson 3.14

Bayes' Theorem

Flipping conditional probabilities — updating beliefs with evidence.

Bayes' Theorem lets you swap the order of conditioning. If you know $P(B \mid A)$ and want $P(A \mid B)$ :

P(A \mid B) = \frac{P(B \mid A)\, P(A)}{P(B)}

The denominator $P(B)$ usually expands via the law of total probability:

P(B) = P(B \mid A)\, P(A) + P(B \mid A^c)\, P(A^c)

The classic worked example: a test for a rare disease (prevalence $1$ in $1{,}000$ ) has $99\%$ sensitivity and $99\%$ specificity. You test positive. What's the probability you actually have the disease?

P(D \mid +) = \frac{P(+ \mid D)\, P(D)}{P(+ \mid D)\, P(D) + P(+ \mid D^c)\, P(D^c)} = \frac{0.99 \cdot 0.001}{0.99 \cdot 0.001 + 0.01 \cdot 0.999} \approx 0.090

Only about $9\%$ . Base rates dominate even very accurate tests when the underlying condition is rare.

This counter-intuitive answer is why Bayesian reasoning shows up everywhere — fraud detection, medical screening, classifier calibration, and any decision where you're updating from a low-prior prior with imperfect evidence.

A spam filter flags $5\%$ of legitimate emails as spam (false positive rate) and catches $95\%$ of actual spam (true positive rate). If $20\%$ of incoming email is actually spam, what is the probability that a flagged email is genuinely spam?