Session 07-05 - Bayes’ Theorem

Section 07: Probability & Statistics

Dr. Nikolai Heinrichs & Dr. Tobias Vlćek

Entry Quiz - 10 Minutes

Quick Review from Session 07-04

Test your understanding of Conditional Probability

  1. If \(P(A) = 0.5\), \(P(B) = 0.4\), and \(P(A \cap B) = 0.2\), find \(P(A|B)\).

  2. A bag has 4 red and 6 blue balls. Two are drawn without replacement. Find \(P(\text{both blue})\).

  3. Given \(P(B|A) = 0.6\) and \(P(A) = 0.3\), find \(P(A \cap B)\).

  4. If \(P(A|B) = P(A)\), what can we conclude about A and B?

Homework Discussion - 12 Minutes

Your Questions from Session 07-04

Let’s clear up conditional probability before Bayes’ reversal.

  • Tree diagram setup and branch probabilities
  • Independence tests using conditional probability
  • Multiplication rule usage

Learning Objectives

What You’ll Master Today

  • Apply Bayes’ Theorem: \(P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}\)
  • Understand classification metrics like true positive/negative rates
  • Compute Positive and Negative Predictive Values from given rates
  • Solve Bayes problems using both the formula and the contingency table method
  • Apply Bayes’ Theorem to medical, business, and other real-world problems

Bayes’ Theorem appears on virtually every Feststellungsprüfung!

Part A: Bayes’ Theorem

Reversing Conditional Probabilities

The problem: We often know \(P(B|A)\) but need \(P(A|B)\).

Example:

  • We know \(P(\text{positive test}|\text{disease})\) (true positive rate)
  • We need \(P(\text{disease}|\text{positive test})\) (positive predictive value)

These are not the same! This is a common misconception.

Bayes’ Theorem Formula

Given \(P(B|A)\), we can compute \(P(A|B)\):

\[P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}\]

Since \(P(B)\) is often not directly known, we expand it using the law of total probability:

\[P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B|A) \cdot P(A) + P(B|A') \cdot P(A')}\]

\(A'\) is the complement of \(A\).

Understanding the Components

A common application is in medical testing:

  • We know how accurate a test is but want to know the probability of actually having a disease given a positive result
Formula Part In a medical test example
\(P(A)\) Probability of having the disease
\(P(B|A)\) Probability of a positive test given disease
\(P(A|B)\) Probability of disease given a positive test
\(P(B)\) Overall probability of a positive test

Introductory Example I

Let’s see Bayes in action with a simple example:

  • In a company, 30% of employees have an MBA degree.
  • Of those with an MBA, 80% were promoted.
  • Of those without an MBA, 40% were promoted.

Question: An employee was promoted. What is the probability they have an MBA?

Introductory Example II

Let’s see Bayes in action with a simple example:

Let \(M\) = has MBA, \(P\) = promoted

\[P(M|P) = \frac{P(P|M) \cdot P(M)}{P(P|M) \cdot P(M) + P(P|M') \cdot P(M')}\]

\[= \frac{0.80 \times 0.30}{0.80 \times 0.30 + 0.40 \times 0.70} = \frac{0.24}{0.24 + 0.28} = \frac{0.24}{0.52} \approx 0.462\]

Makes sense, since 30% of employees have an MBA, and 70% don’t. The probability of being promoted is higher for those with an MBA, but not enough to outweigh the larger group without an MBA.

Tree Diagram for Bayes

Read the joint probabilities from the tree, then divide to reverse the condition!

Part B: Medical Testing Framework

Classification Metrics

Medical tests are a classic application of Bayes’ Theorem, but the same logic applies to any binary classification problem.

Metric Formula Meaning
True Positive Rate \(P(+\|D)\) Prob. of positive given condition
True Negative Rate \(P(-\|D')\) Prob. of negative given no condition
Base Rate \(P(D)\) Proportion with condition in population
Pos. Predictive Value \(P(D\|+)\) Prob. of condition given positive
Neg. Predictive Value \(P(D'\|-)\) Prob. of no condition given negative

These metrics apply wherever you have a 2×2 contingency table — medical tests, fraud detection, spam filters, credit scoring, etc.

Terminology

The same concepts have different names depending on the field. Here is the mapping:

Formula Statistics Medicine Machine Learning
\(P(+\|D)\) True Positive Rate Sensitivity Recall
\(P(-\|D')\) True Negative Rate Specificity —
\(P(D\|+)\) Pos. Predictive Value — Precision
\(P(+\|D')\) False Positive Rate 1 − Specificity Fall-out
\(P(D)\) Base Rate Prevalence Prior

In this course, we use the statistics column. When you encounter synonyms in textbooks or online, map them back to this table.

What Can Go Right and Wrong?

Every test or prediction has four possible outcomes:

  • True Positive: Condition present, correctly predicted positive
  • False Positive: Condition absent, incorrectly predicted positive
  • True Negative: Condition absent, correctly predicted negative
  • False Negative: Condition present, incorrectly predicted negative

Example: COVID Test

A rapid COVID test has:

  • True Positive Rate (Sensitivity): 95%
  • True Negative Rate (Specificity): 98%
  • Base Rate (Prevalence): 2%

Question: If you test positive, what’s the probability you actually have COVID?

Solution Using Bayes’ Theorem

  • \(P(D|+) = \frac{P(+|D) \cdot P(D)}{P(+)}\)

Calculate \(P(+)\) using law of total probability:

  • \(P(+) = P(+|D) \cdot P(D) + P(+|D') \cdot P(D')\)
  • \(= 0.95 \times 0.02 + (1-0.98) \times (1-0.02)\)
  • \(= 0.019 + 0.02 \times 0.98 = 0.019 + 0.0196 = 0.0386\)

Apply Bayes:

  • \(P(D|+) = \frac{0.95 \times 0.02}{0.0386} = \frac{0.019}{0.0386} \approx 0.492\)

Only about 49% of positive tests are true positives when the base rate is low!

Why Low?

Quick Check - 6 Minutes

Bayes Basics

Work individually

  1. A test has a true positive rate of 80% and a true negative rate of 90%. The base rate is 10%. Write Bayes’ formula for the Positive Predictive Value (do not compute).

  2. If \(P(A)=0.3\), \(P(B|A)=0.7\), and \(P(B|A')=0.2\), compute \(P(B)\).

  3. In one sentence: why is \(P(D|+)\) usually much lower than \(P(+|D)\) when the base rate is low?

Break - 10 Minutes

Part C: Systematic Problem-Solving

Step-by-Step Approach

Bayes’ Theorem Strategy

  1. Identify what you need: Usually \(P(A|B)\)
  2. Extract given information: Base rate, true positive/negative rate, false positive/negative rate
  3. Set up the formula: Write Bayes’ theorem
  4. Calculate the denominator using the law of total probability
  5. Substitute and solve: Careful with arithmetic!
  6. Interpret: What does the answer mean in context?

Complete Example: Disease Screening

A screening test for a disease has:

  • True Positive Rate (Sensitivity) = 90%
  • True Negative Rate (Specificity) = 95%
  • Base Rate (Prevalence) = 1%

Find:

  1. Positive Predictive Value
  2. Negative Predictive Value

Solution: Positive Predictive Value I

Find the positive predictive value:

\[P(D|+) = \frac{P(+|D) \cdot P(D)}{P(+|D) \cdot P(D) + P(+|D') \cdot P(D')}\]

Given values:

  • \(P(+|D) = 0.90\) (true positive rate)
  • \(P(D) = 0.01\) (base rate)
  • \(P(+|D') = 1 - 0.95 = 0.05\) (false positive rate)
  • \(P(D') = 0.99\)

Solution: Positive Predictive Value II

Find the positive predictive value:

\[P(D|+) = \frac{0.90 \times 0.01}{0.90 \times 0.01 + 0.05 \times 0.99}\]

\[= \frac{0.009}{0.009 + 0.0495} = \frac{0.009}{0.0585} \approx 0.154\]

The Positive Predictive Value is only 15.4%!

Solution: Negative Predictive Value

Find the negative predictive value:

\[P(D'|-) = \frac{P(-|D') \cdot P(D')}{P(-)}\]

Calculate \(P(-)\):

\[P(-) = P(-|D) \cdot P(D) + P(-|D') \cdot P(D') = 0.9415\]

\[P(D'|-) = \frac{0.95 \times 0.99}{0.9415} = \frac{0.9405}{0.9415} \approx 0.999\]

The Positive Predictive Value is only 15.4%, but the Negative Predictive Value is 99.9%!

The Base Rate Effect

The Positive Predictive Value depends heavily on the base rate. Even a very accurate test produces many false positives when the condition is rare!

Word of Caution

Do Not Mix Up These Three Probabilities:

  • \(P(D \cap -)\): “condition and predicted negative” (joint probability)
  • \(P(D|-)\): “condition given neg. prediction” (conditional probability)
  • \(P(-|D)\): “negative prediction given condition” (false negative rate)

They answer different questions and usually have very different values!

Part D: Contingency Table Method

Alternative Approach

Use a hypothetical population (e.g., 10,000 people):

Condition (+) Condition (−) Total
Predicted + 90 495 585
Predicted − 10 9,405 9,415
Total 100 9,900 10,000

Read directly:

  • Positive Predictive Value = \(\frac{90}{585} = 0.154\)
  • Negative Predictive Value = \(\frac{9405}{9415} = 0.999\)

When to Use Which Method

Both methods are valid, but the table method is often easier:

Formula Method Table Method
Faster for single probability Better when multiple values needed
Direct plug-in Visual and intuitive
Preferred for “show your work” Good for checking answers
Easy to make arithmetic errors Harder to lose track of numbers

On the exam, use the contingency table to verify your Bayes formula result!

Part E: Further Applications

Bayes in Business

Bayes’ Theorem applies whenever you need to reverse a conditional probability!

  • Quality control: defective item came from which machine?
  • Marketing: customer who bought product belongs to which segment?
  • Finance: given a stock rose, was the market trend bullish?
  • Hiring: given a successful employee, which recruitment channel?

Question: What are some other applications of Bayes’ Theorem?

Example: Email Spam Filter

An email filter classifies messages. Historical data shows:

  • 20% of all emails are spam
  • The filter correctly flags 95% of spam emails
  • The filter incorrectly flags 3% of legitimate emails

Question: If an email is flagged as spam, what is the probability it’s actually spam?

\[P(S|F) = \frac{P(F|S) \cdot P(S)}{P(F|S) \cdot P(S) + P(F|S') \cdot P(S')}\]

About 89% of flagged emails are truly spam.

Example: Machine Source

A factory has three machines producing bolts:

  • Machine I: 50% of output, 2% defect rate
  • Machine II: 30% of output, 3% defect rate
  • Machine III: 20% of output, 5% defect rate

Question: A bolt is defective. Which machine most likely produced it?

\[P(D) = 0.02(0.50) + 0.03(0.30) + 0.05(0.20) = 0.029\]

Machine I and III are equally likely — even though III has a higher defect rate, I produces more!

Guided Practice - 20 Minutes

Practice Problem 1

Work in pairs

A factory has two machines:

  • Machine A produces 60% of items, with 3% defect rate
  • Machine B produces 40% of items, with 5% defect rate

If a randomly selected item is defective, what’s the probability it came from Machine A?

Practice Problem 2 (Exam-Style)

Work in pairs

A medical test has a true positive rate of 85% and a true negative rate of 90%.

In a population with a 5% base rate:

  1. Calculate the Positive Predictive Value
  2. Calculate the Negative Predictive Value
  3. Construct a contingency table for 1,000 people
  4. Interpret your results

Practice Problem 3

Work in pairs

In a newborn screening program, “98.9% of all hearing-impaired newborns are recognized by the test” and “4.8% of healthy newborns test positive.”

Let \(D\) = hearing impaired, \(+\) = positive test.

  1. Translate the two statements into probability notation.
  2. If the base rate is \(P(D)=0.0015\), compute \(P(D \cap -)\).
  3. Explain in one sentence why \(P(D \cap -)\) is not the same as \(P(D|-)\).

Practice Problem 4

Work in pairs

A company’s customer base consists of 70% regular customers and 30% new customers. Among regular customers, 15% make a return. Among new customers, 25% make a return.

  1. Compute the overall return rate.
  2. Given a return was made, find the probability the customer is new.

Chained Exam Mini-Problem

Work individually, then compare

A test has a true positive rate of \(0.90\), a false positive rate of \(0.08\), and a base rate of \(0.04\).

  1. Compute \(P(+)\).
  2. Use (a) to compute \(P(D|+)\).
  3. Interpret the result of (b) in one sentence.

Coffee Break - 10 Minutes

Collaborative Problem-Solving - 20 Minutes

Challenge 1: Screening Policy Decision

Think individually (2 min), then work in groups of 3-4

A test has a true positive rate of \(0.92\) and a true negative rate of \(0.94\).

Two populations are considered:

  • Group A base rate: \(1\%\)
  • Group B base rate: \(8\%\)
  1. Compute the Positive Predictive Value for Group A and Group B.
  2. Explain why the values differ even though test quality is unchanged.
  3. Recommend where confirmatory testing is most necessary and justify.

Challenge 2: Insurance Fraud

Think individually, then work in groups

An insurance company flags claims using an automated system:

  • 5% of all claims are fraudulent
  • The system correctly identifies 90% of fraudulent claims
  • The system incorrectly flags 8% of legitimate claims
  1. A claim is flagged. What is the probability it is actually fraudulent?
  2. Construct a table for 2,000 claims and verify your answer.
  3. The company investigates all flagged claims at €500 each. Fraudulent claims cost €10,000 on average. Is the flagging system cost-effective?

Challenge 3: Recruitment Channel

Think individually, then work in groups

A tech company hires through two channels:

  • Online job portals: 70% of hires
  • Employee referrals: 30% of hires

After one year, 60% of portal hires and 85% of referral hires are rated “high performer.”

  1. Draw a tree diagram and compute all joint probabilities.
  2. Compute the overall proportion of high performers.
  3. Given a randomly selected high performer, what is the probability they were a referral hire?

Final Assessment - 5 Minutes

Exit Ticket

Work individually

  1. In one sentence: what is the difference between \(P(+|D)\) and \(P(D|+)\)?
  2. A test has a true positive rate of 0.90, a true negative rate of 0.95, and a base rate of 0.02. Write (do not compute) the formula for the Positive Predictive Value.
  3. Is \(P(D \cap -)\) a joint or conditional probability?

Wrap-Up & Key Takeaways

Today’s Essential Concepts

  • Bayes’ Theorem: \(P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}\)
  • Classification metrics: True positive/negative rate, base rate → Positive/Negative Predictive Value
  • Base rate matters: Low base rate → low Positive Predictive Value (base rate effect)
  • Two methods: Formula or contingency table — use tables to verify!
  • Wide applicability: Medical testing, quality control, spam filters, hiring, insurance

Next Session Preview

Coming Up: Contingency Tables

  • Constructing tables from word problems
  • Reading marginal, joint, and conditional probabilities
  • Independence testing in tables
  • Exam-style problems

Homework

Complete Tasks 07-05 — especially the medical testing and business application problems!