ModelRefs / Bayes' Theorem — Tutorial
Bayes' Theorem — Tutorial
The mathematical foundation of probabilistic reasoning — and how to build a spam filter from scratch. Covers Prior, likelihood, and posterior.
Overview
The mathematical foundation of probabilistic reasoning — and how to build a spam filter from scratch
Level: Intermediate. Estimated reading time: 20 minutes.
Prior, likelihood, and posterior
Bayes' theorem tells you how to update your beliefs when new evidence arrives:
P(A|B) = P(B|A) × P(A) / P(B)
In English: the probability of A given B equals the probability of B given A, times the prior probability of A, divided by the probability of B.
The terms have names: - P(A) — Prior: your belief before seeing evidence - P(B|A) — Likelihood: how probable the evidence is if A is true - P(A|B) — Posterior: your updated belief after seeing evidence - P(B) — Marginal likelihood: probability of the evidence under any hypothesis
Example: a test for a disease is 99% accurate. Only 1% of the population has the disease. You test positive. What's the probability you actually have it?
P(disease | positive) = P(positive | disease) × P(disease) / P(positive) = 0.99 × 0.01 / ((0.99×0.01) + (0.01×0.99)) = 0.0099/0.0198 ≈ 50%
Surprising: even a 99% accurate test leaves you at 50/50 when the disease is rare. This is the base rate fallacy — ignoring how common A is leads to wildly wrong inferences.
Naive Bayes: a fast text classifier
Naive Bayes classifies text by assuming words are conditionally independent given the class — a strong (and often wrong) assumption that nonetheless works remarkably well in practice.
For spam detection, the model learns: - P(spam) — what fraction of emails are spam - P(word | spam) — how often each word appears in spam emails - P(word | ham) — how often each word appears in legitimate emails
To classify a new email, compute: P(spam | words) ∝ P(spam) × Π P(wordᵢ | spam)
And compare to P(ham | words). The class with the higher probability wins.
Naive Bayes is fast (one pass through the training data), interpretable (you can inspect which words drive the spam score), and often competitive with more complex models for short texts. Gmail's early spam filter was essentially Naive Bayes.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Bayes' Theorem — Tutorial.