91Ó°ÊÓ

The likelihood function is a fundamental concept in statistics, particularly in the context of Maximum Likelihood Estimation (MLE). In our given problem, we're considering only two possible outcomes—0 and 1—with associated probabilities \( p \) and \( q \), where \( q = 1 - p \). The likelihood function for a single observation \( x \) is expressed as:

• \( L(p; x) = p^x (1-p)^{1-x} \)

The logic here is straightforward: if \( x = 1 \), the function results in \( p \), and if \( x = 0 \), it results in \( 1-p \). For a sample of \( n \) independent observations, the likelihood function multiplies individual likelihoods:

• \( L(p; x_1, x_2, \ldots, x_n) = \prod_{i=1}^n p^{x_i} (1-p)^{1-x_i} \)

This formulation helps us express what we "likely" observe given a parameter value \( p \). By maximizing this function, we aim to find the value of \( p \) that makes the observed data most probable.

Dealing with the likelihood function's products can be computationally difficult, especially with large samples. To ease this process, we use the natural logarithm to derive the log-likelihood function. This transformation results in sums rather than products:

• \( \ell(p; x_1, x_2, \ldots, x_n) = \sum_{i=1}^n \left( x_i \log(p) + (1-x_i) \log(1-p) \right) \)

Understanding log-likelihood is crucial as it simplifies differentiation. For MLE, maximizing the log-likelihood yields the same parameter estimates as maximizing the likelihood function but is statistically simpler. Specifically, by differentiating the log-likelihood function with respect to \( p \) and setting the derivative to zero, we obtain the MLE for \( p \):

• \( \hat{p} = \frac{\sum_{i=1}^n x_i}{n} \)

Using logs converts multiplicative processes into more manageable additive ones, crucial for easy computation.

An unbiased estimator is a statistic characterized by its expected value being equal to the true parameter value it estimates. In our problem, the MLE of \( p \), which is \( \hat{p} = \frac{\sum_{i=1}^n x_i}{n} \), is unbiased. This means in expectation, it equals \( p \); mathematically expressed as:

• \( E(\hat{p}) = p \)

An unbiased estimator doesn't skew to overestimate or underestimate the parameter based on repeated sampling. It represents an essential property as unbiasedness ensures that there's no systematic error in the estimation process over long-run use. However, just because an estimator is unbiased doesn't mean it has the smallest possible variance, which brings us to consider the precision of estimates further.

The Expected Squared Error (ESE) of an estimator is the unbiasedness measure plus its variance, which provides an indicator of the estimator's efficiency. For our MLE, the ESE is defined as the variance since it is unbiased:

• \( \text{Var}(\hat{p}) = \frac{p(1-p)}{n} \)

However, we need to compare this with another estimator, \( \delta(x) = \frac{1}{2} \), which doesn't change irrespective of data collected. Its ESE accounts solely for the difference from the true parameter:

• \( \text{Err}(\delta) = \left(\frac{1}{2} - p\right)^2 \)

The task is to show that for \( \frac{1}{3} \leq p \leq \frac{2}{3} \), \( \text{Var}(\hat{p}) \geq \text{Err}(\delta) \). This demonstrates that within this range, the constant estimator \( \delta \) sometimes performs better, challenging the typical supremacy of MLE in estimating performance. This underscores that MLE, while powerful, might not always be optimal in terms of expected error across possible parameter values.