Probability distributions part II

Outline

• The following topics will be covered in this lecture:
  • Review of random variables
  • Review of distributions
  • Binomial distribution
  • Parameters of the binomial distribution

Random variables

Courtesy of Ania Panorska CC

Random variables continued

Courtesy of Ania Panorska CC

Random variables continued

Courtesy of Ania Panorska CC

Probability distributions

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

Probability distributions continued

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

Binomial distribution

• A coin flipping experiment is actually a simple example of a broad category of experiments.
• For example, we can consider an experiment in which we describe two possible outcomes:
1. (success / \( S \) / \( 1 \) / \( H \)); or
2. (failure / \( F \) / \( 0 \) / \( T \)).
• We can encode the outcomes any way we like;
• it is common to encode the outcomes as \( S \) or \( F \), where the choice of “success” is arbitrary.
• More generally than coin flipping, we might consider the case where the probabilities, \[ \begin{align} P(\text{success}) \neq P(\text{failure}) \end{align} \]
• Recall, if the experiment has only two possible outcomes, if \( A= \)"success" then \( \overline{A}= \)"failure".
• Therefore, \[ \begin{align} P(\text{success}) + P(\text{failure}) = 1. \end{align} \]
• Suppose we run the experiment a total of \( n \) trials.
• Because there are only two possible outcomes for each trial, and a finite number of trials, we can create a list of all possible outcomes for \( n \) trials.
• For example, if there are \( x_\alpha \) total successes, there must be exactly \( n - x_\alpha \) failures in \( n \) trials.
• More importantly, we can also make a list of all possible ways we can have \( x_\alpha \) successes and \( n-x_\alpha \) failures.
• Let \( x \) be the random variable equal to the number of successful trials – we can therefore calculate the probability, \[ P(x = x_\alpha); \] however, the classical model for probability (equal probability of all outcomes) will no longer apply.

Binomial distribution continued

• Recall our random variable \( x \) equal to the number of successful trials in \( n \) total trials.
• Unlike with coin flipping, we suppose that it is possible for \[ \begin{align} P(\text{success}) \neq P(\text{failure}) \end{align} \]
• However, there are finite trials, finite possible outcomes and, for each possible number of successes \( x_\alpha \), there are a finite number of ways \( x=x_\alpha \).
• Provided all trials are independent (like coin flipping) and the probability of success is constant, we can still make a counting argument using
1. The rule of complementary probability \[ \begin{align} P(\text{success}) + P(\text{failure}) = 1; \end{align} \]
2. independence;
3. the list of all possible ways we can make \( x_\alpha \) successes;
4. the list of all possible ways we can make \( n-\alpha \) failures; and
5. a total of \( n \) trials exactly;
• to compute the probability exactly for each \( x_\alpha \) where \( x_\alpha \) ranges from \( 0, 1, \cdots, n \).
• The list of all possible number of successes \( x_\alpha = 0, 1, \cdots, n \) and the associated probabilities \( P(x= x_\alpha) \) for \( x_\alpha = 0, 1, \cdots, n \) is called the binomial distribution.
• The argument itself is somewhat long, but it really only uses tools we already know.
• Therefore, if you can understand the principles of the points 1 - 5 above, we don’t need to belabor the details in this class.

Binomial distribution continued

• Formally, we will now describe the binomial distribution.
• Suppose we run an experiment with two possible outcomes \( S= \)"success" and \( F= \)"failure", where \[ \begin{align} P(S) = p && P(F) = 1 - P(S) = q. \end{align} \]
• Suppose we run exactly \( n \) total trials of the above experiment and suppose that:
1. each trial is independent; and
2. \( P(S)=p \) for every trial.
• Let \( x \) be the random variable equal to the total number of successful trials.
• Let \( x_\alpha \) be one of the possible number of successful trials in the range \( 0, 1 ,\cdots , n \).
• Then the probability of exactly \( x_\alpha \) successful trials (the event \( x= x_\alpha \)) is given by \[ \begin{align} P(x=x_\alpha) = \frac{n!}{\left( n - x_\alpha\right)! x_\alpha !} p^{x_\alpha} q^{(n - x_\alpha)}, \end{align} \]
1. where the meaning of the “\( ! \)” for any whole number \( m \) is given by \[ \begin{align} m! &= m \times (m-1) \times (m-2) \times \cdots \times 2 \times 1,\\ \end{align} \] i.e., this is the descending product of all whole numbers less than or equal to \( m \) and greater than zero, except for \( 0! = 1 \) which we take as definition.
2. The total number of ways that we can have exactly \( x_\alpha \) successes in \( n \) trials is given by \[ \frac{n!}{\left( n - x_\alpha\right)! x_\alpha !}. \]
3. The probability of \( x_\alpha \) independent successes (or \( n-x_\alpha \) indepednent failures) is \( p^{x_\alpha} \) \( \big( \) or \( q^{(n-x_\alpha)}\big) \) respectively.

Binomial distribution example

• Recall our notation:
  1. \( n \) - the number of trials;
  2. \( x \) - the random variable;
  3. \( x_\alpha \) - a specific number of successes that \( x \) could possibly attain;
  4. \( P(S)= p \) - the probability of an independent trial’s success;
  5. \( P(F)=q \) - the probability of an independent trial’s failure.
• Consider that when an adult is randomly selected with replacement, there is a \( 0.85 \) probability that this person knows what Twitter is (based on results from a Pew Research Center survey).
• Suppose that we want to find the probability that exactly three of five random adults know what Twitter is.
• Discuss with a neighbor: can you identify what \( n \), \( x \), \( x_\alpha \), \( p \) and \( q \) are in the above word problem?
• Here we consider the random selection to be a “trial” so that the number of trials is \( n=5 \)
• If we consider a “successful” trial to be “select an adult who knows what Twitter is”, then \( x \) is “number of adults who know what Twitter is out of five”.
• \( x_\alpha \) is the specific number of successful trials we are interested in, i.e., \( x_\alpha = 3 \).
• \( p \) is the probability of an independent trial’s successs, i.e, \( p=0.85 \)
• \( q \) is the probability of an independent trial’s failure, i.e., \( q=1-p = 0.15 \).

Binomial distribution example continued

• Let’s recall our values from the last slide,
  • Here we consider the random selection to be a “trial” so that the number of trials is \( n=5 \)
  • If we consider a “successful” trial to be “select an adult who knows what Twitter is”, then \( x \) is “number of adults who know what Twitter is out of five”.
  • \( x_\alpha \) is the specific number of successful trials we are interested in, i.e., \( x_\alpha = 3 \).
  • \( p \) is the probability of an independent trial’s successs, i.e, \( p=0.85 \)
  • \( q \) is the probability of an independent trial’s failure, i.e., \( q=1-p = 0.15 \).
• Suppose we wanted to compute the probability of one particular outcome,
• say, \( S_i = \)"the \( i \)-th particpant knows what Twitter is" and \( F_i= \)"the \( i \)-th participant does not know what twitter is", where \[ A = S_1 \text{ and } S_2 \text{ and } S_3 \text{ and } F_4 \text{ and } F_5. \]
• We can use independence and the multiplication rule to show \[ \begin{align} P(A) &= P(S_1)\times P(S_2)\times P(S_3)\times P(F_4)\times P(F_5) \\ &= 0.85 \times 0.85 \times 0.85 \times 0.15 \times 0.15 \\ &= 0.85^3 \times 0.15^2. \end{align} \]
• This shows how we get one part of the binomial distribution formula.
• However, there are many combinations of \( S_i \) and \( F_i \) that arise in \( x=3 \).
• Using a counting argument, we can show that the total number of ways \( x=3 \) is \[ \begin{align} \frac{n!}{(n- x_\alpha)! x_\alpha!} = \frac{5!}{(5 - 3)! 3!} = \frac{5!}{(2)!3!} = 10 \end{align} \]

Binomial distribution example continued

• Let’s recall our values from the last slide,
  • Here we consider the random selection to be a “trial” so that the number of trials is \( n=5 \)
  • If we consider a “successful” trial to be “select an adult who knows what Twitter is”, then \( x \) is “number of adults who know what Twitter is out of five”.
  • \( x_\alpha \) is the specific number of successful trials we are interested in, i.e., \( x_\alpha = 3 \).
  • \( p \) is the probability of an independent trial’s successs, i.e, \( p=0.85 \)
  • \( q \) is the probability of an independent trial’s failure, i.e., \( q=1-p = 0.15 \).
  • The total number of ways \( x=3 \) is \[ \begin{align} \frac{n!}{(n- x_\alpha)! x_\alpha!} = \frac{5!}{(5 - 3)! 3!} = \frac{5!}{(2)!3!} = 10 \end{align} \]
• The binomial distribution formula can then be read as,
• The probability of finding exactly \( 3 \) out of \( 5 \) independently, randomly selected adults who know what Twitter is, is equal to \[ \begin{align} \frac{n!}{(n- x_\alpha)! x_\alpha!} p^{x_\alpha} q^{n- x_\alpha} = 10 \times 0.85^3 \times 0.15^2 \approx 0.138, \end{align} \]
• or in plain English,

  the probability of three independent successful trials, times the probability of two indepdendent failure trials, times all possible ways we can have exactly \( 3 \) successful trials out of five.

• Again, the counting argument can be somewhat long and technical so it will not be the focus of the course,
• however, is important that you understand the pieces of the formula and how it fits together.

Binomial distribution example continued

• Let’s now take a graphical look at the last problem in StatCrunch.
• We should remark the following on the last calculation.
• Technically, we could only make use of the binomial distribution because we sampled with replacement to enforce independent trials.
• If sampled our population without replacement, we know
1. that the trials are dependent; and
2. that the probability of success changes at each trial.
• These conditions make it so the binomial distribution does not apply to the random variable \( x \) when we do not replace samples.
• However it is common to approximate sampling without replacement as independent when the sample size is less than \( 5\% \) of the population.
• In practice for polls of, e.g., all US adults, this approximation will often be used.
• However, in this class, we will only use this approximation when the problem specifically calls for the approximation.

Binomial distribution technology example

• While it is important to understand the pieces and the principles that go into the binomial formula \[ \begin{align} P(x=x_\alpha) = \underbrace{\frac{n!}{\left( n - x_\alpha\right)! x_\alpha !}}_{(1) } \times \underbrace{ p^{x_\alpha}}_{(2)} \times \underbrace{q^{(n - x_\alpha)}}_{(3)} \end{align} \] as:
1. Total number of ways to find exactly \( x_\alpha \) successful trials out of \( n \) total trials;
2. Probability of \( x_\alpha \) independent succesful trials;
3. Probability of \( n-x_\alpha \) independent failure trials;
• in practice, we will usually let technology handle the complicated calculation.
• Because we will let technology handle these calculations, we should understand how the pieces fit together without blindly entering values into formulas.
• We will now consider a more complicated example:
• Before 2012, the NFL used to decide overtime games by a coin flip where the winner could decide if they kicked or recieved the ball in a fresh play.
• Between 1974 and 2011, 460 overtime games did not end in a tie.
• 252 of these games were won by the team that won the coin toss and got to decide whether to kick or recieve.
• Let’s assume that the probability of winning or losing an overtime game is equally likely.
• Discuss with a neighbor: if we want to find the probability of winning exactly \( 252 \) games, can you identify what \( n \), \( x \), \( x_\alpha \), \( p \) and \( q \) are in the above word problem?

Binomial distribution technology example continued

• From the last slide, we will consider, \( n= 460 \) independent trials (overtime games).
• \( x \) is the number of wins in overtime.
• \( x_\alpha=252 \) is the specific number of successful trials we are interested in.
• \( p=q=0.5 \) because (we assume) either outcome is equally likely.
• Therefore, we have, \[ \begin{align} P(x=x_\alpha) &= \frac{n!}{\left( n - x_\alpha\right)! x_\alpha !} \times p^{x_\alpha} \times q^{(n - x_\alpha)}\\ &=\frac{460!}{(460 - 252)!252!} \times 0.5^{252} \times 0.5^{(460 - 252)} \\ &= \frac{460!}{208!252!}\times 0.5^{252} \times 0.5^{208} \end{align} \]
• This is a complicated expression, so therefore we will examine this in StatCrunch directly.

Binomial distribution technology example continued

• Consider the last example where:
• Between 1974 and 2011, 460 overtime games did not end in a tie.
• 252 of these games were won by the team that won the coin toss and got to decide whether to kick or recieve.
• Let’s assume that the probability of winning or losing an overtime game is equally likely.
• Another way of saying this is that,
• we will assume that the result of the coin flip has no impact on whether the team wins or loses.
• Recall, we saw earlier that when observing some outcome, e.g., \( x = 252 \), if the probability of observing some outcome at least as extreme is less than \( 5\% \) we call this interesting or significant.
• If \( 252 \) successful trials is significant, then assuming that winning the coin flip makes no difference on the outcome,
• i.e., assuming \( p=q=0.5 \),
• then we should question this assumption.
• We will look at this in StatCrunch directly – while we do so, Discuss with a neighbor:
• is the \( 252 \) wins significant for the binomial distribution for \( n=460 \) trials and equal probability of success and failure?

Binomial distribution technology example continued

• The process that we took in the last example was what we called before, assuming the null hypothesis.
• We assumed that the probability of winning or losing overtime was equally likely;
• in particular, it shouldn’t depend on the coin toss.
• However, we found that the probability of finding some event at least as extreme was given by, \[ P(x \geq 252) \approx 0.0224 \] in the case that winning or losing is equally likely.
• Being less than \( 5\% \), we called this observation significant, and this strongly suggests that winning the coin flip gave the team an advantage in overtime.
• This is precisely why the overtime rule was changed in 2012.

Parameters of the binomial distribution

Courtesy of Ania Panorska CC

Parameters of the binomial distribution continued

Public domain via Wikimedia Commons

Parameters of the binomial distribution and the range-rule-of-thumb

• Recall that:
• for the binomial distribution the mean is given as, \[ \mu = n \times p . \]
• For the binomial distribution the standard deviation is given as, \[ \sigma = \sqrt{ n \times p \times q} . \]
• Let’s consider the NFL example again, using the range rule of thumb.
• Recall that there were \( 460 \) trials with an assumed probability of \( 0.5 \) for success.
• Discuss with a neighbor: using the range-rule-of-thumb, is \( 252 \) successful trials significant?
• Notice, \[ \begin{align} \mu = 460 \times 0.5 = 230 & & \sigma = \sqrt{460 \times 0.5 \times 0.5} = \sqrt{ 115}\approx 10.72.\end{align} \]
• We call observations that lie outside of the range, \[ (\mu - 2 \sigma , \mu + 2 \sigma ) \approx ( 208.56, 251.44) \] significant.
• The total number of wins \( 252 \) is just outside of the range, so by the range-rule-of-thumb, we also call this a significant observation.

Review of the binomial distribution

• The binomial distribution is a key distribution that gives us a way to model a wide range of experiments probabilistically.
• This applies when we run an experiment with two possible outcomes \( S= \)"success" and \( F= \)"failure", where \[ \begin{align} P(S) = p && P(F) = 1 - P(S) = q. \end{align} \]
• When we run exactly \( n \) total trials of the above experiment, assuming that:
1. each trial is independent; and
2. \( P(S)=p \) for every trial.
• We can model the probability of a particular number of successes \( x_\alpha \) like a (possibly) non-fair coin flipping experiment.
• We model the probability of exactly \( x_\alpha \) successful trials as \[ \begin{align} P(x=x_\alpha) = \underbrace{\frac{n!}{\left( n - x_\alpha\right)! x_\alpha !}}_{(1) } \times \underbrace{ p^{x_\alpha}}_{(2)} \times \underbrace{q^{(n - x_\alpha)}}_{(3)} \end{align} \] where:
1. Total number of ways to find exactly \( x_\alpha \) successful trials out of \( n \) total trials;
2. Probability of \( x_\alpha \) independent succesful trials;
3. Probability of \( n-x_\alpha \) independent failure trials;
• The special structure of this distribution also allows us to compute the mean and standard deviation directly as \[ \begin{align} \mu = n \times p & & \sigma = \sqrt{n \times p\times q} \end{align} \]