91Ó°ÊÓ

The least squares criterion is a fundamental method used in regression analysis. It helps us find the optimal parameters for our model by minimizing the sum of the squared differences between the observed values and the values predicted by the model. These differences, called residuals, measure how well our model fits the data. Specifically, in our multiple regression model, we aim to minimize the sum:
\[ S(\beta_0, \beta_1, \beta_2, \beta_3) = \sum_{i=1}^{n} ( Y_{i} - (\beta_0 + \beta_1 X_{i1} + \beta_2 X_{i1}^{2} + \beta_3 X_{i2}) )^2. \]
This equation tells us that we must adjust our parameters \( \beta_0, \beta_1, \beta_2, \beta_3 \) to make the sum of these squared differences as small as possible.

Once we have defined the least squares criterion, the next step is to derive the normal equations. These equations help us find the optimal parameters by solving a system of linear equations. To derive them, we calculate the partial derivatives of the sum of squares function with respect to each parameter and set these derivatives equal to zero. This process yields the following equations:
\[ -2 \sum_{i=1}^{n} (Y_{i} - \beta_{0} - \beta_{1} X_{i1} - \beta_{2} X_{i1}^{2} - \beta_{3} X_{i2}) = 0, \]
\[ -2 \sum_{i=1}^{n} X_{i1}(Y_{i} - \beta_{0} - \beta_{1} X_{i1} - \beta_{2} X_{i1}^{2} - \beta_{3} X_{i2}) = 0, \]
\[ -2 \sum_{i=1}^n X_{i1}^{2}(Y_{i} - \beta_{0} - \beta_{1} X_{i1} - \beta_{2} X_{i1}^{2} - \beta_{3} X_{i2}) = 0, \]
\[ -2 \sum_{i=1}^n X_{i2}(Y_{i} - \beta_{0} - \beta_{1} X_{i1} - \beta_{2} X_{i1}^{2} - \beta_{3} X_{i2}) = 0. \]
By solving this system of linear equations, we can determine the values of \( \beta_0, \beta_1, \beta_2, \beta_3 \) that minimize the sum of squared residuals.

The likelihood function is a key concept in statistical modeling, used to estimate the parameters of a model. For our regression model, where the errors \( \varepsilon_i \) are normally distributed with mean zero and variance \( \sigma^2 \), the likelihood function is given by:
\[ L(\beta_0, \beta_1, \beta_2, \beta_3, \sigma^2) = \prod_{i=1}^{n} \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left( -\frac{(Y_i - \beta_0 - \beta_1 X_{i1} - \beta_2 X_{i1}^2 - \beta_3 X_{i2})^2}{2\sigma^2} \right). \]
This function represents the probability of observing the given data, assuming the model parameters are correct. By maximizing this function, we find the most likely values of \( \beta_0, \beta_1, \beta_2, \beta_3 \) and \( \sigma^2 \).

To derive the maximum likelihood estimators (MLEs), we seek the parameters that maximize the likelihood function. In practice, we usually minimize the negative log-likelihood because it is easier to work with. For our model, the negative log-likelihood is:
\[ -\log L(\beta_0, \beta_1, \beta_2, \beta_3, \sigma^2) = \frac{n}{2} \log(2\pi) + \frac{n}{2} \log(\sigma^2) + \frac{1}{2\sigma^2} \sum_{i=1}^{n} (Y_i - \beta_0 - \beta_1 X_{i1} - \beta_2 X_{i1}^2 - \beta_3 X_{i2})^2. \]
When we minimize this expression, we obtain the same normal equations we derived earlier using the least squares criterion. Therefore, the MLEs for \( \beta_0, \beta_1, \beta_2, \beta_3 \) are identical to the least squares estimators, confirming that both methods yield the same results.