91Ó°ÊÓ

Residuals are a key component in understanding least-squares regression. When we try to fit a model to data, we often have actual observed values and predicted values from our model. The residual for each data point is simply the difference between these two values. In mathematical terms, for a data point

• Actual value: \(y_i\)
• Predicted value: \(a + bx_i + \frac{c}{x_i}\)

The residual \(r_i\) is calculated as: \[r_i = y_i - (a + bx_i + \frac{c}{x_i})\] Residuals help us understand which data points are far from the fitted line. These differences are crucial as they indicate how well, or poorly, our model predicts the actual data. However, you're not just looking at individual residuals. We are interested in minimizing their sum to find the line that best fits the data overall.

The objective function in least-squares regression is all about minimizing the overall error in your prediction model. This is achieved by focusing on the sum of the squares of the residuals. The goal is straightforward: decrease this sum as much as possible to ensure your model fits the data closely. Mathematically, the objective function \(Q\) is expressed as: \[Q = \sum_{i=1}^5 r_i^2 = \sum_{i=1}^5 \left(y_i - (a + bx_i + \frac{c}{x_i})\right)^2\] The key point here is why we square the residuals. Squaring has a few benefits:

• Eliminates negative values, treating all deviations equally regardless of direction.
• Penalizes larger discrepancies more heavily, as they entail much higher squared values.

Thus, by minimizing this sum of squared residuals, we find model parameters that bring the predicted values closest to the observed data points.

To minimize our objective function effectively, we apply the concept of partial derivatives. Partial derivatives are used to determine how a multivariable function changes as each variable changes while keeping others constant. For least-squares regression, we have a set of parameters (\(a, b, c\)) that we adjust.

• We take the partial derivative of \(Q\) with respect to each parameter.
• Set these derivatives to zero.

Mathematically, this is represented as:

• \(\frac{\partial{Q}}{\partial{a}} = 0\)
• \(\frac{\partial{Q}}{\partial{b}} = 0\)
• \(\frac{\partial{Q}}{\partial{c}} = 0\)

These calculations allow us to find points where the objective function is minimized. Recognizing these points is crucial, as it leads us to the best-fit parameters for our data model.

The partial derivatives render a system of linear equations when set to zero. This system contains equations that must be solved to reach the optimal values of the parameters \(a, b, c\). The equations correspond to conditions where the slopes of the objective function, with respect to each parameter, are zero.To solve this system, different methods are available:

• **Substitution:** Solving one equation for one variable and substituting into others.
• **Gaussian Elimination:** A systematic method for reducing equations to simpler forms.
• **Matrix Inversion:** Using matrix operations to directly solve the equations.

These methods can be computationally intensive, especially with complex models or larger datasets. In practice, software like Excel, R, or Python can simplify the process significantly. By solving these equations, you get the values of \(a, b,\) and \(c\) that minimize the objective function, delivering the best prediction model.