/*! This file is auto-generated */ .wp-block-button__link{color:#fff;background-color:#32373c;border-radius:9999px;box-shadow:none;text-decoration:none;padding:calc(.667em + 2px) calc(1.333em + 2px);font-size:1.125em}.wp-block-file__button{background:#32373c;color:#fff;text-decoration:none} Problem 10 a. Show that \(\sum_{i=1}^{n} e_... [FREE SOLUTION] | 91影视

91影视

Log out

Learning Materials

Features

Discover

Chapter 13: Problem 10

a. Show that \(\sum_{i=1}^{n} e_{i}=0\) when the \(e_{i}\) 's are the residuals from a simple linear regression. b. Are the residuals from a simple linear regression independent of one another, positively correlated, or negatively correlated? Explain. c. Show that \(\sum_{i=1}^{n} x_{i} e_{i}=0\) for the residuals from a simple linear regression. (This result along with part (a) shows that there are two linear restrictions on the \(e_{i}^{\text {'s, resulting }}\) in a loss of 2 df when the squared residuals are used to estimate \(\sigma^{2}\).) d. Is it true that \(\Sigma_{i=1}^{n} e_{i}^{*}=0\) ? Give a proof or a counter example.

Short Answer

Expert verified
a. \( \sum_{i=1}^{n} e_i = 0 \); b. Independent if CLRM assumptions hold; c. \( \sum_{i=1}^{n} x_i e_i = 0 \); d. No, \( \sum_{i=1}^{n} e_i^* = 0 \) is not generally true.

Step by step solution

01

Understand the Residuals Definition

In a simple linear regression model, the residuals are given by \( e_i = y_i - \hat{y}_i \), where \( y_i \) is the actual value and \( \hat{y}_i \) is the predicted value from the regression model. The regression line is given by \( \hat{y}_i = \beta_0 + \beta_1 x_i \), where \( \beta_0 \) and \( \beta_1 \) are the estimated coefficients.
02

Prove the Sum of Residuals Equals Zero

The simple linear regression is fitted by minimizing the sum of squared residuals, which leads to the normal equations. From one of these normal equations, specifically \( \sum_{i=1}^{n} e_i = 0 \), it follows that the sum of the residuals is zero. This is because the partial derivative of the sum of squared residuals with respect to \( \beta_0 \) is set to zero, enforcing this condition as part of the fitting process.
03

Discuss Correlation of Residuals

Residuals in a simple linear regression are assumed to be uncorrelated random variables. If the assumptions of the classic linear regression model (CLRM) are met, particularly that the errors are independently distributed, then the residuals should also be independent. However, any patterns or correlations in residuals could suggest model issues or violations of assumptions.
04

Prove Sum of Weighted Residuals Equals Zero

For simple linear regression, the other normal equation is \( \sum_{i=1}^{n} x_i e_i = 0 \). This equation is derived from minimizing the sum of squared residuals with respect to \( \beta_1 \), which effectively orthogonalizes the residuals to the predictor, thus proving the relation.
05

Evaluate Residual Sum Outside of Canonical Form

\( \sum_{i=1}^{n} e_i^* eq 0 \) generally, where \( e_i^* \) denotes transformations or modifications of the original residuals. Unless \( e_i^* = e_i \) (original residuals), the specific property of summation to zero for residuals does not apply, and modifications can change this outcome.

Unlock Step-by-Step Solutions & Ace Your Exams!

  • Full Textbook Solutions

    Get detailed explanations and key concepts

  • Unlimited Al creation

    Al flashcards, explanations, exams and more...

  • Ads-free access

    To over 500 millions flashcards

  • Money-back guarantee

    We refund you if you fail your exam.

Over 30 million students worldwide already upgrade their learning with 91影视!

Key Concepts

These are the key concepts you need to understand to accurately answer the question.

Residuals
In the context of Simple Linear Regression, residuals are the differences between the observed actual values and the values predicted by the model. Mathematically, a residual is expressed as \( e_i = y_i - \hat{y}_i \), where \( y_i \) represents the actual value and \( \hat{y}_i \) is the predicted value. Residuals play a crucial role in determining the goodness of fit of a regression model. They help identify how well the model captures the underlying data pattern.
When fitting a simple linear regression, an important property of these residuals is that their sum equals zero, i.e., \( \sum_{i=1}^{n} e_i = 0 \). This property follows from one of the normal equations obtained during the regression fitting process. By making the partial derivative of the sum of squared residuals with respect to the intercept \( \beta_0 \) equal to zero, we ensure this characteristic. This condition indicates that the residuals are uniformly spread around the regression line, without any systemic bias.
Sum of Square Residuals
The Sum of Square Residuals (SSR) is a measure that reflects the variation in the actual data which the model fails to capture. It is obtained by adding up the squares of each residual, represented as \( \sum_{i=1}^{n} e_i^2 \).
The reason we square the residuals is to eliminate any potential negative values, ensuring the measure is always positive, and importantly, to place greater emphasis on larger residuals. Squaring emphasizes larger discrepancies between the observed and predicted values, making SSR a valuable metric for assessing the performance of the regression model.
In the process of fitting the linear regression model, minimizing the Sum of Square Residuals is crucial. By minimizing this value, we effectively derive the best fitting line through the dataset, represented through the ordinary least squares methodology. This is accomplished using the normal equations, ensuring that the final model parameters \( \beta_0 \) and \( \beta_1 \) are estimators that minimize the SSR.
Normal Equations
Normal equations are vital components in deriving the coefficients of a simple linear regression model. These equations arise from the method of minimizing the sum of squared residuals, a key goal in regression analysis. By setting the partial derivatives of the SSR with respect to the coefficients to zero, we form the normal equations:
  • \( \sum e_i = 0 \)
  • \( \sum x_i e_i = 0 \)
These conditions ensure that the residuals are balanced and orthogonal to the predictors. The first normal equation, \( \sum e_i = 0 \), implies that the mean of the residuals is zero, ensuring that the model does not systematically underestimate or overestimate the observed values.
The second equation, \( \sum x_i e_i = 0 \), reflects the concept that the residuals should not have any linear association with the predictor variables. This orthogonality condition ensures that the derived regression line is indeed the line of best fit, leading to more reliable and unbiased parameter estimates. Through these equations, normal equations guarantee the optimality of the linear regression solution.

One App. One Place for Learning.

All the tools & learning materials you need for study success - in one app.

Get started for free

Most popular questions from this chapter

The article "Bank Full Discharge of Rivers" (Water 91影视 \(\left.J_{.}, 1978: 1141-1154\right)\) reports data on discharge amount \(\left(q\right.\), in \(\left.\mathrm{m}^{3} / \mathrm{sec}\right)\), flow area \(\left(a\right.\), in \(\left.\mathrm{m}^{2}\right)\), and slope of the water surface \((b\), in \(\mathrm{m} / \mathrm{m})\) obtained at a number of floodplain stations. A subset of the data follows. Let \(y=\ln (q), x_{1}=\ln (a)\), and \(x_{2}=\ln (b)\). Consider fitting the model \(Y=\beta_{0}+\beta_{1} x_{1}+\beta_{2} x_{2}+\epsilon\). a. The resulting \(h_{i i}\) 's are \(.138, .302, .266, .604, .464\), \(.360, .215, .153, .214\), and \(.284\). Does any observation appear to be influential? b. The estimated coefficients are \(\hat{\beta}_{0}=1.5652, \hat{\beta}_{1}=\) \(.9450\), and \(\hat{\beta}_{2}=.1815\), and the corresponding estimated standard deviations are \(s_{\hat{\beta}_{1}}=.7328, s_{\hat{\beta}_{t}}=\) \(.1528\), and \(s_{\dot{H}_{2}}=.1752\). The second standardized residual is \(e_{2}^{*}=2.19\). When the second observation is omitted from the data set, the resulting estimated coefficients are \(\hat{\beta}_{0}=1.8982, \hat{\beta}_{1}=1.025\), and \(\hat{\beta}_{2}=.3085\). Do any of these changes indicate that the second observation is influential? c. Deletion of the fourth observation (why?) yields \(\hat{\beta}_{0}=1.4592, \hat{\beta}_{1}=.9850\), and \(\hat{\beta}_{2}=.1515\). Is this observation influential?

Curing concrete is known to be vulnerable to shock vibrations, which may cause cracking or hidden damage to the material. As part of a study of vibration phenomena, the paper "Shock Vibration Test of Concrete" (ACI Materials \(J ., 2002: 361-370\) ) reported the accompanying data on peak particle velocity ( \(\mathrm{mm} / \mathrm{sec})\) and ratio of Transverse cracks appeared in the last 12 prisms, whereas there was no observed cracking in the first 18 prisms. a. Construct a comparative boxplot of ppv for the cracked and uncracked prisms and comment. Then estimate the difference between true average ppv for cracked and uncracked prisms in a way that conveys information about precision and reliability. b. The investigators fit the simple linear regression model to the entire data set consisting of 30 observations, with ppv as the independent variable and ratio as the dependent variable. Use a statistical software package to fit several different regression models,

The article cited in Exercise 49 of Chapter 7 gave summary information on a regression in which the dependent variable was power output \((\mathrm{W})\) in a simulated 200 -m race and the predictors were \(x_{1}=\) arm girth \((\mathrm{cm}), x_{2}=\) excess post-exercise oxygen consumption \((\mathrm{ml} / \mathrm{kg})\), and \(x_{3}=\) immediate posttest lactate (mmol/L). The estimated regression equation was reported as $$ \begin{aligned} &y=-408.20+14.06 x_{1}+.76 x_{2}-3.64 x_{3} \\ &\left(n=11, R^{2}=.91\right) \end{aligned} $$ a. Carry out the model utility test using a significance level of .01. [Note: All three predictors were judged to be important.] b. Interpret the estimate \(14.06\). c. Predict power output when arm girth is \(36 \mathrm{~cm}\), excess oxygen consumption is \(120 \mathrm{ml} / \mathrm{kg}\), and lactate is \(10.0\). d. Calculate a point estimate for true average power output when values of the predictors are as given in (c). e. Obtain a point estimate for the true average change in power output associated with a \(1 \mathrm{mmol} / \mathrm{L}\) increase in lactate while arm girth and oxygen consumption remain fixed.

Does exposure to air pollution result in decreased life expectancy? This question was examined in the article "Does Air Pollution Shorten Lives?" (Statistics and Public Policy, Reading, MA, Addison-Wesley, 1977). Data on $$ \begin{aligned} y &=\text { total mortality rate }(\text { deaths per } 10,000) \\ x_{1} &=\text { mean suspended particle reading }\left(\mu \mathrm{g} / \mathrm{m}^{3}\right) \\ x_{2} &=\text { smallest sulfate reading }\left(\left[\mu \mathrm{g} / \mathrm{m}^{3}\right] \times 10\right) \\ x_{3} &=\text { population density }\left(\text { people } / \mathrm{mi}^{2}\right) \\ x_{4} &=\text { (percent nonwhite) } \times 10 \\ x_{5} &=\text { (percent over } 65) \times 10 \end{aligned} $$ for the year 1960 was recorded for \(n=117\) randomly selected standard metropolitan statistical areas. The estimated regression equation was $$ \begin{aligned} y=& 19.607+.041 x_{1}+.071 x_{2} \\ &+.001 x_{3}+.041 x_{4}+.687 x_{5} \end{aligned} $$ a. For this model, \(R^{2}=.827\). Using a \(.05\) significance level, perform a model utility test. b. The estimated standard deviation of \(\hat{\beta}_{1}\) was 016 . Calculate and interpret a \(90 \%\) CI for \(\beta_{1}\). c. Given that the estimated standard deviation of \(\hat{\beta}_{4}\) is \(.007\), determine whether percent nonwhite is an important variable in the model. Use a .01 significance level. d. In 1960 , the values of \(x_{1}, x_{2}, x_{3}, x_{4}\), and \(x_{5}\) for Pittsburgh were \(166,60,788,68\), and 95 , respectively. Use the given regression equation to predict Pittsburgh's mortality rate. How does your prediction compare with the actual 1960 value of 103 deaths per 10,000 ?

The viscosity \((y)\) of an oil was measured by a cone and plate viscometer at six different cone speeds \((x)\). It was assumed that a quadratic regression model was appropriate, and the estimated regression function resulting from the \(n=6\) observations was $$ y=-113.0937+3.3684 x-.01780 x^{2} $$ a. Estimate \(\mu_{\gamma \cdot 75}\), the expected viscosity when speed is \(75 \mathrm{rpm} .\) b. What viscosity would you predict for a cone speed of \(60 \mathrm{rpm}\) ? c. If \(\Sigma y_{i}^{2}=8386.43, \Sigma y_{i}=210.70, \Sigma x_{i} y_{i}=17,002.00\), and \(\Sigma x_{i}^{2} y_{i}=1,419,780\), compute SSE \(\left[=\Sigma y_{i}^{2}-\right.\) \(\left.\hat{\beta}_{0} \Sigma y_{i}-\hat{\beta}_{1} \Sigma x_{i} y_{i}-\hat{\beta}_{2} \Sigma x_{i}^{2} y_{i}\right]\) and \(s\). d. From part (c), SST \(=8386.43-(210.70)^{2} / 6=987.35\). Using SSE computed in part (c), what is the computed value of \(R^{2}\) ? e. If the estimated standard deviation of \(\hat{\beta}_{2}\) is \(s_{\hat{\beta}_{2}}=.00226\), test \(H_{0}: \beta_{2}=0\) versus \(H_{2}: \beta_{2} \neq 0\) at level \(.01\), and interpret the result.
See all solutions

Recommended explanations on Math Textbooks

Statistics

Read Explanation

Geometry

Read Explanation

Applied Mathematics

Read Explanation

Calculus

Read Explanation

Decision Maths

Read Explanation

Probability and Statistics

Read Explanation
View all explanations

What do you think about this solution?

We value your feedback to improve our textbook solutions.

Study anywhere. Anytime. Across all devices.