Continuous random variables and univariate distributions

Outline

• The following topics will be covered in this lecture:
• A review of the basics of continuous distributions
• The uniform distribution
• The normal distribution

A review of continuous random variables

• Unlike discrete random variables, continuous random variables can take on an uncountably infinite number of possible values.

  • This is to say that if \( X \) is a continuous random variable, there is no possible index set \( \mathcal{I}\subset \mathbb{Z} \) which can enumerate the possible values \( X \) can attain.
  • For discrete random variables, we could perform this with a possibly infinite index set, \( \{x_j\}_{j=1}^\infty \)
  • This has to do with how the infinity of the continuum \( \mathbb{R} \) is actually larger than the infinity of the counting numbers, \( \aleph_0 \);
  • in the continuum you can arbitrarily sub-divide the units of measurement.

• These random variables are characterized by a distribution function and a density function.

• Let , then the mapping \[ F_X:\mathbb{R} \rightarrow [0,1] \] defined by \( F_X (x) = P(X \leq x) \), is called the cumulative distribution function (cdf) of the rv \( X \).

• A mapping \( f_X: \mathbb{R} \rightarrow \mathbb{R}^+ \) is called the probability density function (pdf) of an rv \( X \) if \( f_X(x) = \frac{\mathrm{d} F_X}{\mathrm{d}x} \) exists for all \( x\in \mathbb{R} \); and

• and the density is integrable, i.e., \[ \int_{-\infty}^\infty f_X (x) \mathrm{d}x \] exists and takes on the value one.

A review of continuous random variables

• Q: we defined, \[ \begin{align} f_X(x) = \frac{\mathrm{d} F_X}{\mathrm{d}x} & & \text{ and }& & \int_{a}^b \frac{\mathrm{d}f}{\mathrm{d}x} \mathrm{d}x = f(b) - f(a) \end{align} \] how can you use the definition above and the fundamental theorem of calculus to find another form for the CDF?

  • A: Notice that \( \frac{\mathrm{d} F_X}{\mathrm{d}x} \) means that the CDF can be written in terms of the anti-derivative of the density.
  • If \( s \) and \( t \) are arbitrary values, the definite integral is written as

  \[ \begin{align} \int_{s}^t f_X(x) \mathrm{d}x &= \int_{s}^t \frac{\mathrm{d} F_X}{\mathrm{d}x} \mathrm{d}x\\ &= F_X(t) - F_X(s) \\ & = P(X \leq t) - P(X \leq s) = P(s < X \leq t) \end{align} \]

  • If we take a limit as \( s \rightarrow \infty \) we thus recover that

  \[ \begin{align} \lim_{s\rightarrow - \infty} \int_{s}^t f_X(x) \mathrm{d} x & = \lim_{s \rightarrow -\infty} P(s < X \leq t) \\ & = P(X\leq t) = F_X(t) \end{align} \]

Properties of continuous distributions

• Last week we discussed how the elementary properties of the probability distribution of a discrete rv can be described by an expectation and a variance.
• With respect to this, the only difference with continuous rvs is in the use of integrals, rather than sums, over the possible values of the rv.
• Let \( X \) be a continuous rv with a density function \( f_X(x) \) – then the expectation of \( X \) is defined as \[ \mathbb{E}\left[X\right] = \int_{-\infty}^{+\infty} xf_X(x)\mathrm{d}x = \mu_X \] where \( f_X \) is the density function described before.
• Note that the same interpretation of the expected value from discrete rvs applies here:
1. We see \( \mathbb{E}\left[X\right]=\mu_X \) as representing the “center of mass” for the “density” curve \( f_X \).
2. We see \( \mathbb{E}\left[X\right]=\mu_X \) as representing the mean that we would obtain if we could take infinitely many independently replicated measurements of \( X \), and took the average of these measurements over all possible scenarios.
• If the expectation of \( X \) exists, the variance is defined as \[ \begin{align} \mathrm{var} \left(X\right)& = \mathbb{E}\left[\left(X − \mu_X \right)^2\right] \\ &=\int_{-\infty}^\infty \left(x - \mu_X\right)^2 f_X(x)\mathrm{d}x = \sigma_X^2 \end{align} \]

• Once again, this is a measure of dispersion by averaging the deviation of each case from the mean in the square sense, weighted by the probability density.

Properties of continuous distributions

• While the variance is a more “fundamental” theoretical quantity for various reasons, in practice we are usually concerned with the standard deviation of the random variable \( X \), \[ \mathrm{std}(X)=\sqrt{\mathrm{var}\left(X\right)} = \sigma_X. \]
• This is due to the fact that the variance \( \sigma^2_X \) has the units of \( X^2 \) by the definition of the product.
• For example, if the units of \( X \) are \( \mathrm{cm} \), then \( \sigma_X^2 \) will be in \( \mathrm{cm}^2 \).
• Taking a square root on the variance gives us the standard deviation \( \sigma_X \) in the units of \( X \) itself.

Quantiles / percentiles

• While together the mean \( \mu_X \) and the standard deviation \( \sigma_X \) give a picture of the center and dispersion of a probability distribution, we can analyze this in a different way.

• For example, while the mean is the notion of the “center of mass”, we may also be interested in where the upper and lower \( 50\% \) of values are separated as a different notion of “center”.

  • The value that separates this upper and lower half does not need to equal the center of mass in general, and it is known commonly as the median.

• More generally, for any univariate cumulative distribution function \( F \), and for \( 0 < p < 1 \), we can identify \( p \) as a percent of the data that lies under the graph of a density curve.

  • We might be interested in where the lower \( p \) area is separated from the upper \( 1-p \) area.

• The quantity \[ \begin{align} F^{-1}(p)=\inf \left\{x \vert F(x) \geq p \right\} \end{align} \] is called the theoretical \( p \)-th quantile or percentile of \( F \).

• The “\( \inf \)” in the above refers to the smallest possible quantity in the set on the right-hand-side.

• We will usually refer to the \( p \)-th quantile as \( \xi_p \).

• \( F^{-1} \) is called the quantile function.

  • Particularly, \( \xi_{-\frac{1}{2}} \) is known as the theoretical median of a distribution.

Skewness and kurtosis