# Activity 09/02/2020

## STAT 757 -- Section 1001 <br> Instructor: Colin Grudzien<br>

## Instructions:
We will work through the following series of activities as a group and hold small group work and discussions in Zoom Breakout Rooms.  Follow the instructions in each sub-section when the instructor assigns you to a breakout room.

## Activities:

### Activity 1: Deriving the normal equations

The "normal" equations are derived exactly in terms of the objective function we studied earlier that measures the sum of square deviations of the proposed mean response from the observed cases:
$$J\left(\overline{\beta}_0,\overline{\beta}_1\right) = \sum_{i=1}^n \left(Y_i - \overline{\beta}_0 - \overline{\beta}_1 X_i\right)^2,$$</li>


Note that $\hat{\beta}_0,\hat{\beta}_1$ are **defined** as the unique minimizers of $J$, and are therefore critical values for the function. Use this information to derive the normal equations for $\hat{\beta}_0,\hat{\beta}_1$:
  $$\begin{align}
  \sum_{i=1}^n Y_i &= n\hat{\beta}_0 + \hat{\beta}_1 \sum_{i=1}^n X_i \\
  \sum_{i=1}^n X_i Y_i & =  \sum_{i=1}^n \left(\hat{\beta}_0 X_i +  \hat{\beta}_1 X_i^2\right)
  \end{align}$$


### Activity 2: properties of the residuals
Use the definition of the normal equations
  $$\begin{align}
  \sum_{i=1}^n Y_i &= n\hat{\beta}_0 + \hat{\beta}_1 \sum_{i=1}^n X_i ,\\
  \sum_{i=1}^n X_i Y_i & =  \sum_{i=1}^n \left(\hat{\beta}_0 X_i +  \hat{\beta}_1 X_i^2\right),
  \end{align}$$
  and the definition of the residuals,
  $$\begin{align} \hat{\epsilon}_i &= Y_i - \hat{Y}_i ,\end{align}$$
  to prove the following statements:
  <ol>
    <li>The sum of the residuals is zero, i.e.,
    $$\sum_{i=1}^n \hat{\epsilon}_i = 0$$</li>
    <li>The sum of the observed values equals the sum of the fitted values,
    $$\sum_{i=1}^n Y_i = \sum_{i=1}^n \hat{Y}_i$$</li>
    <li>The sum of the residuals, weighted by the corresponding predictor variable, is zero,
    $$\sum_{i=1}^n X_i \hat{\epsilon}_i = 0$$</li>
    <li>The sum of the residuals, weighted by the corresponding fitted value, is zero,
    $$\sum_{i=1}^n \hat{Y}_i \hat{\epsilon}_i = 0$$</li>
    <li>The estimated regression function always passes through the point defined by the means (or the center of mass), $\left(\overline{X},\overline{Y}\right)$.
    </ol>


