91Ó°ÊÓ

In linear regression, the design matrix, commonly denoted by \( X \), plays an essential role. It organizes the input data into a structured format that allows operations like multiplication and transposition necessary for further calculations. For the simplest form of linear regression, like when estimating the mean \( \mu \) as presented in the exercise, the design matrix is a column vector filled with ones.

• For \( n \) observations, the design matrix is a vector of size \( n \times 1 \).
• This vector enables us to multiply the "constant" parameter \( \mu \) for each observation.
• The form simplifies many calculations when dealing with linear equations.

For instance, given \( y = X \mu + \epsilon \), the matrix \( X \) simply holds the ones to allow multiplication by \( \mu \), aligning each observed outcome \( y_i \) with \( \mu \). This is why it's vital in expressing models in matrix terms, preparing them for further statistical analysis.

The least squares estimation method is a cornerstone in data analysis. It’s a statistical method used to find the parameter estimates that minimize the difference between observed and predicted values. In linear regression, this means finding the line of best fit.

• The least squares estimator \( \widehat{\mu} \) is given by \( (X^T X)^{-1} X^T y \).
• For our exercise, this simplifies to the mean \( \bar{y} \), showing how much least squares can simplify estimations.
• This approach ensures that the sum of the squared differences between observed and predicted values is as small as possible.

You can think of it as finding the "average" position of the data, which is why, in simple cases like this, it corresponds precisely to the arithmetic mean. The elegance of least squares is in its ability to extend to more complex models while maintaining simplicity and comprehensibility.

The normal distribution is a fundamental concept in statistics, often described by its bell-shaped curve. It's characterized by two parameters: the mean \( \mu \) and the variance \( \sigma^2 \). In the context of this exercise, assuming that the data comes from a normal distribution implies the following:

• The outcomes \( y_1, y_2, \ldots, y_n \) are centered around a true mean \( \mu \).
• Each outcome deviates from \( \mu \) based on a normal distribution with variance \( \sigma^2 \).
• The errors or disturbances \( \epsilon_i \) are normally distributed, \( \epsilon_i \sim N(0, \sigma^2) \).

This distribution assumption allows us to apply various statistical methodologies, like the calculation of \( \widehat{\mu} \) and variance, by leveraging properties of the normal distribution, such as symmetrical tails and defined sample behavior around the mean. In practical terms, assuming normality provides a basis for developing robust statistical inferences.

Sample variance is a measure of how data points differ from the mean. It gives insights into the spread and variability of the data. In the context of this exercise, the sample variance \( s^2 \) describes how the observed values \( y_j \) deviate from their average \( \bar{y} \).

• Mathematically, it is calculated as \( \frac{1}{n-1} \sum_{j=1}^{n} (y_{j} - \bar{y})^{2} \).
• The term \( n-1 \) is used to provide an unbiased estimate of the population variance, commonly referred to as "degrees of freedom."
• This variance estimate helps to assess how well the calculated mean represents the data.

Understanding sample variance is crucial because it affects how we interpret the precision of \( \widehat{\mu} \), which is pivotal when making inferences about the entire population based on the sample data. Hence, sample variance provides a powerful tool for quantifying uncertainty.