- Chapter 13
- Probability
- 23 questions
Probability
Conditional probability, the multiplication theorem, independent events, total probability and Bayes' theorem.
This chapter is about what happens to a probability when you learn something. Conditional probability makes that precise, and everything else follows from it.
Bayes' theorem is the chapter's destination. It reverses the direction of reasoning: given that an effect has been observed, how likely is each possible cause? That is exactly the question a medical test, a quality inspection or a diagnostic system asks, which is why case-based questions in this chapter so often describe one.
What you should be able to do
- Compute conditional probabilities and apply the multiplication theorem.
- Test whether two events are independent, and distinguish independence from mutual exclusivity.
- Apply the theorem of total probability to a partitioned sample space.
- Apply Bayes' theorem to find the probability of a cause given an observed outcome.
Topics
5 topics
- Conditional Probability
The definition of, and how conditioning changes the sample space.
- Multiplication Theorem on Probability
The multiplication rule, and its extension to three or more events.
- Independent Events
The test for independence, and why independent and mutually exclusive are different things.
- Theorem of Total Probability
Partitioning a sample space and combining the conditional probabilities across it.
- Bayes' Theorem
Reversing conditional probability to find the probability of a cause from an observed effect.
Formulas
8 items · All formulas
Conditional probability
P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}- Valid when
- P(B) \neq 0.
Multiplication theorem
P(A \cap B) = P(B)\,P(A \mid B) = P(A)\,P(B \mid A)Independent events
P(A \cap B) = P(A)\,P(B)Equivalently P(A\mid B) = P(A) — knowing B changes nothing about A.
Theorem of total probability
P(A) = \sum_{i=1}^{n} P(E_i)\,P(A \mid E_i)- Valid when
- E_1, \dots, E_n must be mutually exclusive and exhaustive.
Bayes' theorem
P(E_i \mid A) = \dfrac{P(E_i)\,P(A \mid E_i)}{\displaystyle\sum_{j=1}^{n} P(E_j)\,P(A \mid E_j)}Complement
P(A') = 1 - P(A)Addition rule
P(A \cup B) = P(A) + P(B) - P(A \cap B)The subtraction avoids counting the overlap twice; it vanishes for mutually exclusive events.
At least one of two independent events
P(\text{at least one}) = 1 - P(A')\,P(B')- Valid when
- A and B independent.
Key points
6 items · All key points
P(A \mid B) and P(B \mid A) are different — the event already known goes after the bar.
Independent and mutually exclusive are not the same thing, and for events of non-zero probability they are incompatible.
Mutually exclusive means P(A\cap B) = 0; independent means P(A\cap B) = P(A)P(B), which is non-zero when both are.
In total probability and Bayes' theorem, the causes must be mutually exclusive and exhaustive.
Check that the prior probabilities sum to 1.
The denominator in Bayes' theorem is the total probability of the observed event.
Name the events in words before writing any numbers — most errors in this chapter are set-up errors.
For 'at least one', it is nearly always faster to compute the complement.
Common mistakes
5 items · All common mistakes
- Mistake
Interchanging P(A \mid B) with P(B \mid A).
InsteadThe condition — the event known to have occurred — is written after the bar. Identify it before writing the fraction.
WhyBoth quantities involve the same two events, and word problems often describe them in the reverse order.
- Mistake
Treating mutually exclusive events as independent.
InsteadIf A and B are mutually exclusive with non-zero probabilities, they are dependent: knowing A occurred tells you B did not.
WhyBoth terms suggest 'unrelated' in ordinary speech.
- Mistake
Using only one branch in the denominator of Bayes' theorem.
InsteadThe denominator is the total probability of the observed event — sum over every cause.
WhyThe numerator is computed first and the denominator is then written to match it.
- Mistake
Applying total probability when the listed causes do not cover every possibility.
InsteadCheck that the prior probabilities sum to 1 before proceeding.
- Mistake
Multiplying probabilities of dependent events as though they were independent.
InsteadUse P(A\cap B) = P(A)P(B\mid A) unless independence has been established or stated.
WhyMultiplication is the reflex for 'and', and the conditional adjustment is forgotten.