Conditional Probability
What is Conditional Probability?
Conditional probability is about calculating the probability of an event happening given that another event has already happened. The key idea is that new information can change the probability of an event.
For example, suppose you are wondering about the probability that today is humid. That is some number. However, if you find out that yesterday it was raining, the probability that today is humid changes. That updated probability is a conditional probability.
Coin Example
In the previous lesson, we established that the probability of obtaining two heads when you throw two coins works as follows. The first coin can be heads or tails, and for each of those two options, the second coin can also be heads or tails. That gives us four possibilities:
| First Coin | Second Coin | Outcome |
|---|---|---|
| H | H | HH |
| H | T | HT |
| T | H | TH |
| T | T | TT |
The sample space has size 4 and the event (both heads) has size 1, so:
\[ P(HH) = \frac{1}{4} \]
Now consider a different question: what is the probability that both coins land on heads if you know that the first one landed on heads?
With this new information, the bottom half of the table (where the first coin is tails) no longer applies. Our new sample space contains only the outcomes where the first coin is heads:
| First Coin | Second Coin | Outcome |
|---|---|---|
| H | H | HH |
| H | T | HT |
The denominator is no longer 4 but 2, and we still have one favorable outcome (HH). So the answer is:
\[ P(HH \mid \text{first is H}) = \frac{1}{2} \]
This is called a conditional probability. We denote it with a vertical bar. The expression \(P(HH \mid \text{first is H})\) reads as “the probability of heads-heads, given that the first is heads.”
What if the First Coin is Tails?
Now consider: what is the probability of landing heads twice given that the first coin landed on tails?
If the first coin landed on tails, our sample space is now restricted to:
| First Coin | Second Coin | Outcome |
|---|---|---|
| T | H | TH |
| T | T | TT |
The denominator is 2, but the numerator is zero because there is no HH outcome in this restricted sample space:
\[ P(HH \mid \text{first is T}) = \frac{0}{2} = 0 \]
This makes sense. If one of the coins landed on tails, there is no way both coins can be heads. The condition changed the probability from \(\frac{1}{4}\) all the way to zero.
Conditioning on an event restricts the sample space. We only consider outcomes that are compatible with the condition, and then compute probabilities within that reduced space.
Dice Example
Here is a question from the previous lesson: what is the probability that the sum of two dice is 10?
Looking at the \(6 \times 6\) grid of all outcomes, there are three favorable pairs: (4,6), (5,5), and (6,4). So:
\[ P(\text{sum} = 10) = \frac{3}{36} = \frac{1}{12} \]
Conditioning on the First Die Being 6
Now suppose you are told that the first die landed on 6. What is the probability that the sum is 10?
Because the first die is 6, we now live only in the bottom row of the grid (where the first die is 6). That row has 6 outcomes, and only one of them gives a sum of 10: the pair (6, 4).
\[ P(\text{sum} = 10 \mid \text{first} = 6) = \frac{1}{6} \]
The probability changed from \(\frac{1}{12}\) to \(\frac{1}{6}\). Knowing that the first die is 6 made a sum of 10 more likely.
Conditioning on the First Die Being 1
What if the first die landed on 1? Now we live in the top row of the grid (where the first die is 1). That row has 6 outcomes, but none of them produce a sum of 10 (the maximum possible sum with a first die of 1 is \(1 + 6 = 7\)).
\[ P(\text{sum} = 10 \mid \text{first} = 1) = \frac{0}{6} = 0 \]
If the first die is 1, there is no way to reach a sum of 10 because the second die can be at most 6.
General Product Rule
Recall the product rule for independent events from the previous lesson:
\[ P(A \cap B) = P(A) \cdot P(B) \]
This only works when \(A\) and \(B\) are independent. When they are not independent, we need a more general formula.
Deriving the General Rule
Consider this question: what is the probability that the first die lands on 6 and the sum is 10?
The sample space has 36 pairs. The only favorable outcome is (6, 4), because if the first die is 6 and the sum is 10, the second die must be 4.
\[ P(\text{first} = 6 \cap \text{sum} = 10) = \frac{1}{36} \]
Now let us try to express this as a product. The probability that the first die is 6 is:
\[ P(\text{first} = 6) = \frac{6}{36} = \frac{1}{6} \]
What do we multiply this by to get \(\frac{1}{36}\)? We need the probability that the sum is 10 among the outcomes where the first die is 6. There are 6 such outcomes, and only 1 of them (the pair (6,4)) gives a sum of 10:
\[ P(\text{sum} = 10 \mid \text{first} = 6) = \frac{1}{6} \]
Putting it together:
\[ P(\text{first} = 6) \cdot P(\text{sum} = 10 \mid \text{first} = 6) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36} \]
This matches what we calculated directly.
General Formula
The general product rule is:
\[ P(A \cap B) = P(A) \cdot P(B \mid A) \]
This formula works every time, whether events are independent or not.
When events are independent, knowing that \(A\) happened does not help you predict \(B\). In that case, \(P(B \mid A)\) is the same as \(P(B)\), and the formula becomes:
\[ P(A \cap B) = P(A) \cdot P(B) \]
That is the simpler product rule you saw in the previous lesson. The general version just replaces \(P(B)\) with \(P(B \mid A)\) to account for situations where \(A\) does affect \(B\).
Review Questions
1. You roll two dice. What is the probability that both show 6, given that the first die shows 6?
If the first die is 6, the sample space reduces to 6 outcomes (the first die is fixed). Only one of those gives (6,6). So \(P(\text{both 6} \mid \text{first} = 6) = \frac{1}{6}\).
1. A bag contains 4 red and 6 blue marbles. You draw one marble and it is red. What is the probability that a second draw (without replacement) is also red?
After removing one red marble, the bag contains 3 red and 6 blue marbles (9 total). So \(P(\text{second red} \mid \text{first red}) = \frac{3}{9} = \frac{1}{3}\).
1. If \(P(A) = 0.3\) and \(P(B \mid A) = 0.5\), what is \(P(A \cap B)\)?
Using the general product rule: \(P(A \cap B) = P(A) \cdot P(B \mid A) = 0.3 \times 0.5 = 0.15\).
1. Explain why \(P(HH \mid \text{first is T}) = 0\) using the concept of a reduced sample space.
When we condition on the first coin being tails, the sample space reduces to only the outcomes where the first coin is T: {TH, TT}. Neither of these outcomes is HH, so there are zero favorable outcomes in the reduced sample space. Therefore the probability is \(\frac{0}{2} = 0\).
Dependent vs Independent Events
Let us apply the product rule with a concrete example to understand when events are dependent and when they are independent.
Random Room Assignment (Independent)
In a school with 100 kids, 50 play soccer and 50 do not. We split them into two rooms of size 50 randomly. Because the split is random, knowing that a kid plays soccer does not help you predict which room they went to. These two events (plays soccer, is in room 1) are independent.
World Cup Room (Dependent)
Now change the setup. Room 1 has a TV showing the World Cup, and room 2 has a TV showing an unrelated movie. How many kids in room 1 do you expect to like soccer?
Intuitively, all the kids who like soccer will go to the room showing the World Cup. The events “plays soccer” and “is in room 1” are now dependent, because knowing that a kid plays soccer makes it much more likely they chose room 1. When events are dependent, the probabilities change compared to the independent case.
Applying the Product Rule
Consider a school of 100 kids where 40 play soccer and 60 do not. Among the kids who play soccer, 80% of them wear running shoes on any given day. How many kids play soccer and wear running shoes?
We can estimate this directly: 80% of 40 is 32. So about 32 kids play soccer and wear running shoes.
Now let us use the product rule to get the same answer:
- \(P(S) = 0.4\) (40% play soccer)
- \(P(R \mid S) = 0.8\) (80% of soccer players wear running shoes)
\[ P(S \cap R) = P(S) \cdot P(R \mid S) = 0.4 \times 0.8 = 0.32 \]
So the probability that a random kid plays soccer and wears running shoes is 32%.
Non-Soccer Players
Now suppose somebody tells us that among the kids who do not play soccer, 50% wear running shoes. What is the probability that a kid does not play soccer and wears running shoes?
- \(P(S') = 0.6\) (60% do not play soccer)
- \(P(R \mid S') = 0.5\) (50% of non-soccer players wear running shoes)
\[ P(S' \cap R) = P(S') \cdot P(R \mid S') = 0.6 \times 0.5 = 0.30 \]
So 30% of kids do not play soccer but still wear running shoes.
Probability Tree
A probability tree is a visual way to organize all the possible outcomes and their probabilities. For the soccer and running shoes example, the tree looks like this:
There are four scenarios in total:
| Scenario | Probability |
|---|---|
| Plays soccer and wears running shoes | \(0.4 \times 0.8 = 0.32\) |
| Plays soccer and does not wear running shoes | \(0.4 \times 0.2 = 0.08\) |
| Does not play soccer and wears running shoes | \(0.6 \times 0.5 = 0.30\) |
| Does not play soccer and does not wear running shoes | \(0.6 \times 0.5 = 0.30\) |
Notice that all four probabilities add up to 1.00 (which makes sense, because these four cases cover every possibility).
Recognizing Dependent vs Independent Events
Let us recap the different cases we have seen:
- Random room split: Kids are assigned to rooms at random. Knowing whether a kid plays soccer tells you nothing about which room they are in. The events are independent.
- World Cup room: One room shows the World Cup. Knowing that a kid plays soccer makes it very likely they chose that room. The events are dependent.
- Soccer and running shoes: Kids who play soccer are more likely (80%) to wear running shoes than kids who do not (50%). Knowing that a kid plays soccer changes the probability of them wearing running shoes. The events are dependent.
The key test: if knowing that event \(A\) happened changes the probability of event \(B\), then \(A\) and \(B\) are dependent. If it does not change the probability, they are independent.
Review Questions
1. A software company conducted a test on their new platform by exposing users to two versions of the same product. Version A: 4000 users. Version B: 5000 users. Total users who experienced a bug: 3000. Users with Version B who experienced a bug: 1500. What is the probability that a user tested Version B, given they experienced a bug?
Using \(P(X \mid Y) = \frac{P(X \cap Y)}{P(Y)}\). Total users = 9000. \(P(\text{B} \cap \text{bug}) = \frac{1500}{9000}\). \(P(\text{bug}) = \frac{3000}{9000}\). So \(P(\text{B} \mid \text{bug}) = \frac{1500/9000}{3000/9000} = \frac{1500}{3000} = 0.5 = 50\%\).
1. In a room, there are 200 people: 30 people only like soccer, 100 people only like basketball, and 70 people like both soccer and basketball. What is the probability that a randomly selected person likes basketball given they like soccer?
- \(\frac{7}{20}\)
- \(\frac{1}{2}\)
- \(\frac{3}{7}\)
- \(\frac{7}{10}\)
d. \(\frac{7}{10}\). People who like soccer = 30 (only soccer) + 70 (both) = 100. Of those 100 soccer fans, 70 also like basketball. \(P(B \mid S) = \frac{P(B \cap S)}{P(S)} = \frac{70/200}{100/200} = \frac{70}{100} = \frac{7}{10}\).