Fundamentals of probability part II
02/20/2020
Addition rule
Courtesy of Bin im Garten CC via Wikimedia Commons
- Suppose we want to compute the probability of two events \( A \) and \( B \) joined by the compound operation “or”.
- We read the statement, \[ P(A \text{ or } B) \] as he probability of event:
- \( A \) occuring,
- event \( B \) occuring, or
- both \( A \) and \( B \) ocurring.
- Intuitively, we can express the probability in terms of all the ways \( A \) can occur and all the ways \( B \) can occur, if we don’t double count.
- Let all the ways that \( A \) can occur be represented by the red circle to the left.
- Let all the ways that \( B \) can occur be represented by the dashed circle to the left.
- Discuss with a neighbor: suppose we count all the ways \( A \) can occur and all the ways \( B \) can occur.
- If we take the sum of the total of all ways \( A \) occurs and the total of all ways \( B \) occurs, does this give the total of all ways \( A \) or \( B \) occurs?
- Consider, if there is an overlap where both \( A \) and \( B \) occur simultaneously, then summing the total of all ways \( A \) occurs and the total of all ways \( B \) occurs double counts the the cases where both \( A \) and \( B \) occur.
Addition rule continued
Courtesy of Bin im Garten CC via Wikimedia Commons
- Let us consider the statement \[ {\color{red} {P(A)}} + P(B), \] is equal to the sum of the total of all ways that \( A \) occurs and the total of all ways that \( B \) occurs, relative to all possible outcomes.
- Discuss with a neighbor: what term “\( \cdot \)” is needed in \( P\left( \cdot \right) \) below to eliminate the double counting? \[ P(A\text{ or }B)= {\color{red} {P(A)}} + P(B) - P\left( \cdot \right) \]
- We count the cases where \( A \) and \( B \) both occur twice, as these cases are included in both \( {\color{red} {P(A)}} \) and \( P(B) \).
- Therefore, the addition rule for compound events is given as, \[ P(A\text{ or }B) = P(A) + P(B) - P(A\text{ and }B) \]
- Discuss with a neighbor: notice that if \( P(A\text{ and } B) = 0 \) then \[ P(A\text{ or }B) = P(A) + P(B), \] is an accurate statement.
- If \( P(A\text{ and } B) = 0 \), what does this say about the relationship between \( A \) and \( B \)?
- This says that events \( A \) and \( B \) never occur simultaneously.
- An easy example is for \( A= \)"coin flip lands heads" and \( B= \)"coin flip lands tails" – these two events never occur simultaneously and \( P(A\text{ or } B)= P(A) + P(B) \).
- Two events that never occur sumultaneously are called disjoint or mutually exclusive events, corresponding to when there is no overlap.
Addition rule continued
Courtesy of Bin im Garten CC via Wikimedia Commons
- Actually, not only are the events \( A= \)"coin flip lands heads" and \( B= \)"coin flip lands tails" disjoint, but they are complementary.
- If our process is one coin flip, we can encode the simple events as \( \{H,T\} \) for heads and tails respectively.
- Thus \( A=\{H\} \) and \( B=\{T\} \), so that \[ \overline{A}=``\text{all outcomes where the coin flip is not heads''}=\{T\}=B. \]
- Complementary events are special cases of disjoint events.
- We saw last time that, \[ P(A) + P\left(\overline{A}\right) = 1. \]
- We can now use the addition rule to show that this is always true.
- First note that by definition, \( A \) and \( \overline{A} \) will never occur – this is the joined event where \( A \) occurs and \( A \) does not occur simultaneously, so that \( P(A\text{ and }\overline{A})= 0 \).
- However, \( A \) or \( \overline{A} \) is the joint event in which \( A \) occurs, \( A \) does not occur, or both \( A \) occurs and \( A \) does not occur – this is every possible outcome (and some that are impossible). Therefore, \( P\left(A\text{ or }\overline{A}\right) = 1 \)
- The addition rule thus gives us, \[ 1= P\left(A\text{ or }\overline{A}\right) = P(A) + P\left(\overline{A}\right) \] because they are disjoint.
Addition rule example
Courtesy of Bin im Garten CC via Wikimedia Commons
- Let us consider an example about how to use the statement, \[ P(A) + P\left(\overline{A}\right) = 1 \] effectively to deduce probabilities.
- Based on the article,
“Prevalance and Comorbidity of Nocturnal Wandering in the US General Population,”
- we say the probability of randomly selecting a US adult who has sleepwalked is \( 0.292 \).
- Discuss with a neighbor: what is the probability of randomly selecting a US adult who has not sleepwalked?
- Note that if our process is “Randomly selected person” and \( A= \)"person has sleepwalked", then \( \overline{A}= \)"person has not sleepwalked".
- Setting up the problem in this way we can see that \[ \begin{align} & P(A) + P\left(\overline{A}\right) = 1 \\ \Leftrightarrow & P\left(\overline{A}\right) = 1 - P(A) \\ \Leftrightarrow & P\left(\overline{A}\right) = 1 - 0.292 = .708 \end{align} \]
Addition rule example
Courtesy of Mario Triola, Essentials of Statistics, 5th edition
- In the table to the left, we see results from a pre-employment drug screening test with \( 1000 \) participants.
- The results are from a quick test that is innacurate and doesn’t always give correct results.
- Specifically we have four different scenarios for a test result:
- True positive – a subject uses drugs and the test result is positive
- False positive – a subject does not use drugs and the test result is positive.
- True negative – a subject does not use drugs and the test result is negative.
- False negative – a subject uses drugs and the test result is negative.
- Discuss with a neighbor: if one of the participants are selected randomly, let the events be defined as \( A= \)"the participant is a drug user" or \( B= \)"the test result is positive". What is \( P(A\text{ or }B) \)?
- Notice that \( A \)=“True positive or False negative” and that \( B= \)"True positive or False positive". Therefore, \( A \) and \( B \) is the case “True positive” and we have
- \[ \begin{matrix} P(A) = \frac{44\text{ True positive } + 6\text{ False negative}}{1000} & P(B) = \frac{44\text{ True positive } + 90\text{ False positive}}{1000} & P(A\text{ and }B) = \frac{44\text{ True positive}}{1000} \end{matrix} \]
- Thus we see how we would double count \( A \) and \( B \) occuring with \( P(A) + P(B) \) and instead we have,
\[ \begin{matrix}P(A\text{ or }B) = \frac{44\text{ True positive } + 6\text{ False negative} + 90\text{ False positive}}{1000}=0.14\end{matrix} \]
Multiplication rule
Courtesy of Bin im Garten CC via Wikimedia Commons
- There are several ways to consider the multiplication rule for probability – the “physics” way to consider this, due to Kolmogorov, is as follows:
- Suppose that there are two related events \( A \) and \( B \) where knowledge of one occuring would change how likely we see the other to occur.
- For example, we can say \( A= \)"it snows in the Sierra" and \( B= \)"it rains in my garden".
- The day before, I don’t know if either will occur.
- However, if I knew that \( A \) occured, this would change how likely it would seem that \( B \) occurs;
- \( B \) is not guaranteed when \( A \) occurs, but the probability of \( B \) occuring would be higher in the presence of \( A \).
- Supose that \( A \) occurs hypothetically, then our sample space of possible events now only includes events where \( A \) also occurs.
- I.e., we would need to restrict our consideration of \( B \) relative to the case that \( A \) occurs.
- We define the probability of \( B \) conditional on \( A \),
\[ P(B\vert A), \]
as the probability of \( B \) in the case that \( A \) occurs.
Multiplication rule continued
Courtesy of Bin im Garten CC via Wikimedia Commons
- We now consider the probability of \( B \) conditional on \( A \),
\[ P(B\vert A), \]
as the probability of \( B \) in the case that \( A \) occurs. - For example, we can say \( A= \)"it snows in the Sierra" and \( B= \)"it rains in my garden".
- Assuming \( A \) occurs, we will consider all ways for both \( A \) and \( B \) to occur.
- The sample space for \( B \vert A \) has been restricted to the cases where \( A \) occurs, so we compute the probability relative to all the ways \( A \) occurs.
- Therefore the probability of \( P(B\vert A) \) can be read, \[ \frac{\text{All the ways }A\text{ and }B\text{ can occur}}{\text{All the ways }A\text{ can occur}} \]
- Mathematically we write this as, \[ P(B\vert A) = \frac{P(A\text{ and } B)}{P(A)}. \]
- For example, in plain English we can say
The probability that it rains in my garden, given that it snows in the Sierra, is equal to the probability of both occuring relative to the probability of snow in the Sierra.
Multiplication rule continued
Courtesy of Bin im Garten CC via Wikimedia Commons
- We have now defined the conditional probability of \( B \) given \( A \) as, \[ P(B\vert A)=\frac{P(A\text{ and }B)}{P(A)} \] in the Kolmogorov way.
- We should make some notes about this:
- The above statment only makes sense when \( P(A)\neq 0 \), because we can never divide by zero.
- “Physically” we can interpret the meaning with \( P(B\vert A) \) read as
The probability that \( B \) occurs given that \( A \) occurs.
- The above should not be defined when \( A \) is impossible – the phrase “given that \( A \) occurs” makes no sense.
- Using the definition of conditional probability, we get the multiplication rule.
- The multiplication rule for probability tells us that, \[ P(A \text{ and } B) = P(B\vert A) \times P(A) \]
- We will use the above formula quite generally, but we note:
- \( P(B\vert A) \) is not defined when \( P(A)=0 \)
- However, \( P(A\text{ and }B) = 0 \) when \( P(A)=0 \) because this is the probability that \( B \) and the impossible event \( A \) both occur.
Sampling and Independence continued
Courtesy of Mario Triola, Essentials of Statistics, 5th edition
- We define the two concepts we discussed on the last slide below:
- Sampling with replacement – selections are re-introduced to the sample pool before the next sample is made.
- In this case, the selections are independent events.
- Sampling without replacement – selections are not re-introduced to the sample pool before the next sample is made.
- In this case, the selections are dependent events.
- Consider the pre-employment drug screening again, with data above.
- Discuss with a neighbor: suppose we sample two participants without replacement.
- Let \( A= \)"first draw uses drugs" and \( B= \)"second draw uses drugs".
- If we want to compute the probability that both particpants use drugs, what event are we trying understand, \[ \begin{matrix} (A \textbf{ and } B) & or & (A \textbf{ or } B)? \end{matrix} \]
- Can we use the addition rule or the multiplication rule?
- We are considering the event \( (A \textbf{ and } B) \) because we specified both \( A \) and \( B \) occur.
- Therefore, we will use the multiplication rule.
Sampling and Independence continued
Courtesy of Mario Triola, Essentials of Statistics, 5th edition
- For the events
- \( A= \)"first draw uses drugs", and
- \( B= \)"second draw uses drugs",
- we can compute \( P(A\text{ and } B) \) using the multiplication rule as follows:
- Note that \( P(A) \) can be computed as, \[ \frac{50 \text{ Participants using drugs}}{1000 \text{ Participants}} \]
- And that \( P(B\vert A) \) can be computed as, \[ \frac{49\text{ Remaining participants using drugs}}{999 \text{ Remaining participants}} \]
- Therefore, we can write, \[ \begin{align} P(A\text{ and } B)&= P(B\vert A) \times P(A)\\ &= \frac{49}{999} \times \frac{50}{1000} \\ &\approx 0.0024 \end{align} \]