91Ó°ÊÓ

In the context of weighted least squares, variance plays a key role in understanding how data spreads around the mean. Variance is essentially a measure of how much the observed data can diverge from the expected value. In a standard least squares approach, each observation is assumed to have the same variance. However, in weighted least squares, the variance depends on the level of the independent variable, denoted as \(x\).

The formula given for variance is \( V\left(Y_{i} \mid x_{i}\right)=\sigma_{i}^{2}=\frac{\sigma^{2}}{w_{i}} \). Here, \(w_i\) are constants known as weights, highlighting the importance of variance that changes with \(x\). This variation implies that some observations have higher reliability due to lower variance, hence weighted more in the analysis. To manage this variance-dependent weighting, we incorporate these weights into the objective function to ensure that each observation contributes appropriately based on its variance.

Normal equations are derived by setting the derivative of the objective function to zero, aiming to find the values of the coefficients, \(\beta_0\) and \(\beta_1\), that minimize the sum of squared errors. When working with weighted least squares, the objective function becomes:

• \( \sum_{i=1}^{n} w_i (y_i - \beta_0 - \beta_1 x_i)^2 \)

This modified objective function when differentiated with respect to the parameters \(\beta_0\) and \(\beta_1\) provides us the normal equations:

\( \hat{\beta}_0 \sum_{i=1}^{n} w_i + \hat{\beta}_1 \sum_{i=1}^{n} w_i x_i = \sum_{i=1}^{n} w_i y_i \) and \( \hat{\beta}_0 \sum_{i=1}^{n} w_i x_i + \hat{\beta}_1 \sum_{i=1}^{n} w_i x_i^2 = \sum_{i=1}^{n} w_i x_i y_i \).

These equations are essential as they ensure the weighted residuals are as small as possible. Solution to these equations provides the linear model’s slope and intercept, adjusted for the variability within the data.

The use of matrix representation for solving equations is very efficient, especially with multiple variables and large datasets. In weighted least squares, the normal equations can be expressed in a compact matrix form:

\[\begin{bmatrix}\sum_{i=1}^{n} w_i & \sum_{i=1}^{n} w_i x_i \\sum_{i=1}^{n} w_i x_i & \sum_{i=1}^{n} w_i x_i^2\end{bmatrix}\begin{bmatrix}\hat{\beta}_0 \\hat{\beta}_1\end{bmatrix}= \begin{bmatrix}\sum_{i=1}^{n} w_i y_i \\sum_{i=1}^{n} w_i x_i y_i \end{bmatrix}\]

This format is beneficial as it simplifies the computation by leveraging matrix operations. The system of equations becomes easier to handle when written in this matrix notation, allowing the use of linear algebra techniques. To solve for \(\hat{\beta}_0\) and \(\hat{\beta}_1\), we simply multiply both sides by the inverse of the coefficient matrix. This yields the parameter estimates that are optimized to reflect the variance and weight considerations in the data.

The weighted least squares estimators \(\hat{\beta}_0\) and \(\hat{\beta}_1\) offer a refined method for estimating regression coefficients that account for variance variability. Unlike unweighted least squares, these estimators adjust to give observations with lower variance a larger influence.

The explicit formulas for these estimators are:

• \(\hat{\beta}_1 = \frac{\sum_{i=1}^{n} w_i (x_i - \overline{x}_w) y_i}{\sum_{i=1}^{n} w_i (x_i - \overline{x}_w)^2}\)
• \(\hat{\beta}_0 = \overline{y}_w - \hat{\beta}_1 \overline{x}_w \)

Where:

• \( \overline{x}_w = \frac{\sum_{i=1}^{n} w_i x_i}{\sum_{i=1}^{n} w_i} \)
• \( \overline{y}_w = \frac{\sum_{i=1}^{n} w_i y_i}{\sum_{i=1}^{n} w_i} \)

These weighted averages \(\overline{x}_w\) and \(\overline{y}_w\) reflect the central tendency of the data adjusted for variance. These calculations ensure that the resulting regression line is well adapted to the data’s variability, providing reliable predictions even when variance is not constant.