Normal probability distributions part II

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• So far we have learned two sets of skills
1. Summary statistics – used for analyzing samples; and
2. Probability – used to analyze complex events abstractly.
• Our goal is to use statistics from small, representative samples to say something general about the larger, unobservable population.
• Recall, the measures of the population are what we referred to as parameters.

• Parameters are generally unknown and unknowable.
• For example, the mean age of every adult living in the United States is a parameter for the adult population of the USA.
• We cannot possibly know this value exactly as there are people who cannot be surveyed and / or don’t have accurate records.
• If we have a representative sample we can compute the sample mean.
• The sample mean will almost surely not equal population mean, due to the natural variation (sampling error) that occurs in any given sample.
• However, if we have a good probabilistic model for the ages of adults, we can use the sample statistic to estimate the general, unknown population parameter.

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• Let’s consider a hypothetical example from the book.
• Autonomous vehicles are a new and controversial technology;
• let’s suppose as in the book that as a population parameter, \( 70\% \) of US adults do not feel comfortable being driven by an autonomous vehicle.
• In a TE Connectivity survey of \( 1000 \) US adults, \( 69\% \) of this sample responded that they also do not feel comfortable riding in an autonomous vehicle.
• We will suppose that this survey is repeated \( 50,000 \) times to verify the accuracy of the sample statistic with replication.
• Each time the survey results are replicated, we obtain a slightly different value for the proportion due to the inherent variability (sampling error).

• The histogram above shows the sample-based values for the proportion of adults who do not feel comfortable riding in an autonomous vehicle.
• Consider the following: what is interesting about the shape of the histogram above? What do you notice about how the sample-proportions are centered with respect to the population parameter \( 70\% \)?
• The above histogram is shaped like a normal distribution.
• Moreover, the center of the histogram lies very close to the true population parameter \( 70\% \).
• This hypothetical example illustrates an important general property that will allow us to estimate population parameters.
• When we do not know the true population parameter, its value can be estimated by representative samples.
• Knowing the distribution of repeated sampling, we can estimate how close our sample value is to the true parameter.

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• Let’s consider the last example more generally.
• Suppose there is some population and we wish to find the true proportion \( p \) of the population for which some statement is true.
• In the last example,
• the population was US adults,
• the statement was “Uncomfortable being diven by an autonomous vehicle”
• the true proportion was \( p=0.70 \).
• Let’s suppose that we will draw exactly \( n \) observations of the population by random sampling.

• In the last example, the number of observations was \( n=1,000 \) adults.
• Suppose we want to replicate this sampling procedure infinitely many times…
• It impossible to replicate the sampling infinitely many times, but we can construct a probabilistic model for this replication process with a probability distribution.
• Formally, we will define \( \hat{p} \) to be the random variable equal to the proportion derived from a random sample of \( n \) observations.
• For each replication, \( \hat{p} \) attains a different value based on chance.
• Then, for random, independent samples, \( \hat{p} \) tends to be normally distributed about \( p \).
• We can thus use the value of \( \hat{p} \) and the distribution of \( \hat{p} \) to estimate \( p \) and how close we are to it.

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• From the last slide, we will denote the random variable \( \hat{p} \) to be the sample proportion.
• The way this random variable behaves over infinitely many replications is described by the sampling distribution for sample proportions.
• Let’s consider another concrete example – let the sampling procedure be given by \( 5 \) rolls of a fair, six-sided die.
• The random variable \( \hat{p} \) will be the proportion of five observations that are odd.

• All values for each observation \( 1, 2, 3, 4, 5,6 \) are equally likely with half even and half odd.
• Therefore, we can compute \( p \) – the population proportion – exactly as \( p=0.5 \).
• That is, over infinitely many rolls of the dice, we expect half of the rolls to be odd.
• However, each time we roll a fair, six-sided die \( 5 \) times we will obtain a different value for the random variable \( \hat{p} \).
• Example sample proportions \( \hat{p} \) which are attained over different samples are pictured in the middle panel above.
• If we replicate the sampling procedure many times, we can get a picture of the sampling distribution of sample proportions;
• In the right panel above, we see the result of \( 10,000 \) replicated sampling procedures – i.e., rolling a fair, six-sided die \( 5 \) times, repeated \( 10,000 \) times.
• The distribution is approximately normal, with center at the true value \( p=0.5 \).

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• We may also consider how to estimate a population mean \( \mu \).
• Suppose there is some population and there is some numerical measure of the population \( x \) that we wish to find the true population mean \( \mu \) for.
• For example,
• the population can be all US adults,
• the numerical measure can be \( x= \)"age"
• the true mean would be the average age of all US adults.
• Let’s suppose that we will draw exactly \( n \) observations of the population by random sampling.

• Suppose we want to replicate this sampling procedure infinitely many times…
• It is once again impossible to replicate the sampling infinitely many times, but we can construct a probabilistic model for this replication process with a probability distribution.
• Formally, we will define \( \overline{x} \) to be the random variable equal to the mean derived from a random sample of \( n \) observations.
• For each replication, \( \overline{x} \) attains a different value based on chance.
• Then, for large numbers of random, independent samples, \( \overline{x} \) tends to be normally distributed about \( \mu \).
• We can thus use the value of \( \overline{x} \) and the distribution of \( \overline{x} \) to estimate \( \mu \) and how close we are to it.
• NOTE: one key difference from the sample proportions is that we need large sample sizes \( n \) for the distribution of sample means to be “close-to-normal”.

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• We will begin with a very simplified example;
  • usually the population size will be greater, and the number of observations in a sample should be much, much larger.
• We will only use this example because we can look at the sampling distribution of the means concretely.
• Suppose that the population of interest is a household with three children;
• we want want to find the mean age of the population, where the children’s ages are \( 4, 5, \) and \( 9 \).
• Our sampling procedure is to select two children with replacement and ask their age .
• If \( x \) is the random variable equal to the age of a particular observation, \( \overline{x} \) will be the sample mean over two randomly chosen observations.
• To the upper left, we have a table of all possible combinations of observations of the population with a sample size \( n=2 \).
• In the middle column, we have the sample mean \( \overline{x} \) corresponding to the two observations.
• Each possible combination for the observations are all equally likely, so they all have probability \( \frac{1}{9} \).
• If we combine all possible ways we can obtain \( \overline{x}=4.0 \), all possible ways \( \overline{x}=4.5 \), etc… we have the bottom table.
• The bottom table associates a probability value to each possible value \( \overline{x} \) might attain – therefore this is the probability distribution for the sample means \( \overline{x} \).

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• Let’s recall, we have a special name for the mean of a probability distribution:
• this is denoted the expected value.
• Consider the following: with the three ages in the population being \( 4,5, \) and \( 9 \) what is the population mean \( \mu \)? What is the expected value of the probability distribution for the sample means?
• Notice that, \[ \mu = \frac{4 + 5 + 9}{3} = 6. \]
• On the other hand, we have the expected value equal to,

\[ \begin{align} &\sum_{\overline{x}_\alpha \in \mathbf{R}} \overline{x}_\alpha \times P(\overline{x} = \overline{x}_\alpha )\\ =&4.0 \times P(\overline{x}=4.0) + 4.5 \times P(\overline{x}=4.5) + 5.0 \times P(\overline{x}=5.0) + 6.5 \times P(\overline{x}=6.5) + 7.0 \times P(\overline{x}=7.0) + 9 \times P(\overline{x} = 9.0)\\ =&4.0 \times\frac{1}{9} + 4.5 \times \frac{2}{9} + 5.0 \times \frac{1}{9} + 6.5 \times \frac{2}{9} + 7.0 \times \frac{2}{9} + 9 \times\frac{1}{9}\\ =&6.0 \end{align} \]
• That is, the expected value of the sample mean \( \overline{x} \) is equal to the population mean \( \mu \).
• Put another way, over infinitely many replicated samples we would expect to find the true population mean by taking the average over all the replicates;
• this occurs generally, even when the probability distribution for the sample mean is not very normal as above.

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• Suppose we consider rolling the fair, six-sided die again.
• The sampling procedure will once again be to roll the die \( 5 \) times.
• \( x \) will be the random variable that will attain the value of a single observation in the sample.
• \( \overline{x} \) will be the mean of the observations in a given sample (5 rolls).
• By calculating all the possible outcomes, we can find that \( \mu=3.5 \), i.e.,
  • over infinitely many rolls of the dice, we expect to get an average value of \( 3.5 \).

• Each time we replicate the sampling procedure (each time we roll the die \( 5 \) times) we obtain a different value for \( \overline{x} \).
• In the middle panel of the figure, we have possible values for \( \overline{x} \) that we record.
• On the right panel, we see the result from replicating the sampling procedure \( 10,000 \) times.
• In this case, the distribution of the sample means \( \overline{x} \) is approximately normal.
• Also, as will always be the case, the sample means \( \overline{x} \) are distributed around the true population mean \( \mu = 3.5 \).

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• We will now consider how to estimate a population variance \( \sigma^2 \).
• Suppose there is some population and there is some numerical measure of the population \( x \) that we wish to find the true population variance of \( \sigma^2 \) for.
• For example,
• the population can be all US adults,
• the numerical measure can be \( x= \)"age"
• the true variance would be the variance in the ages.
• Let’s suppose that we will draw exactly \( n \) observations of the population by random sampling.

• Suppose we want to replicate this sampling procedure infinitely many times…
• We can construct a probabilistic model for this replication process with a probability distribution.
• Formally, we will define \( s^2 \) to be the random variable equal to the variance derived from a random sample of \( n \) observations.
• For each replication, \( s^2 \) attains a different value based on chance.
• Then, for random, independent samples, \( s^2 \) tends to be distributed, right-skewed about \( \sigma^2 \).
• We can thus use the value of \( s^2 \) and the distribution of \( s^2 \) to estimate \( \sigma^2 \) and how close we are to it.
• NOTE: the primary difference with the distribution of sample variances from the previous examples is that this distribution is not normal, though still has a mean equal to \( \sigma^2 \).

Courtesy of Mario Triola, Essentials of Statistics, 6th edition

• Suppose we consider rolling the fair, six-sided die again.
• The sampling procedure will once again be to roll the die \( 5 \) times.
• \( x \) will be the random variable that will attain the value of a single observation in the sample.
• \( s^2 \) will be the variance of the observations in a given sample (5 rolls).
• By calculating all the possible outcomes, we can find that \( \sigma^2=2.9 \), i.e.,
  • over infinitely many rolls of the dice, we expect to get a population variance of \( 2.9 \).

• Each time we replicate the sampling procedure (each time we roll the die \( 5 \) times) we obtain a different value for \( s^2 \).
• In the middle panel of the figure, we have possible values for \( s^2 \) that we record.
• On the right panel, we see the result from replicating the sampling procedure \( 10,000 \) times.
• In this case, the distribution of the sample variances \( s^2 \) is distributed right-skewed about \( \sigma^2 \).
• As will always be the case, the sample variances have an expected value of \( \sigma^2 \), but due to the skewness of the distribution, the mean does not equal the mode.

Courtesy of Mathieu Rouaud CC BY-SA via Wikimedia Commons

• The property of the sample means \( \overline{x} \) being approximately normally distributed (for large enough sample sizes) is actually a fundamental and universal property in nature.
• In fact, the random variable \( x \) can be extremely non-normal as pictured on the left of the figure.
• Nonetheless, if we draw \( n \) independent samples for \( n \) sufficiently large,
• the sample mean \( \overline{x} \) will be approximately normally distributed around the true \( \mu \).
• We also know about “how far” a realization of \( \overline{x} \) usually lies from \( \mu \) by the standard devaition of \( \overline{x} \).

• Formally, this phenomena is called the Central limit theorem:

Let \( x \) be a generic random variable with population mean \( \mu \) and standard deviation \( \sigma \). Suppose for a sample size of \( n \), we compute the sample mean as \( \overline{x} \). Then \( \overline{x} \) as a random variable, with realization determined by independent replicated sampling, will be approximately normally distributed with mean \( \mu \) and standard deviation \( \frac{\sigma}{\sqrt{n}} \), so long as \( n \) is sufficiently large.

• For many situations, \( n \) needs to only be at least be greater than \( 30 \) for this to hold – however, this is not necessary if \( x \) is already normal.
• Notice, as \( n \) becomes large, the approximation improves and \( \frac{\sigma}{\sqrt{n}} \) becomes small.
• Therefore, for large sample sizes \( n \), \( \overline{x} \) tends to give estimates of \( \mu \) which are close because the standard deviation is small.