91Ó°ÊÓ

Maximum Likelihood Estimation (MLE) is a statistical method used to estimate the parameters of a given probability distribution, by maximizing a likelihood function. The likelihood function represents the probability of observing the given data under various parameter values. To find a maximum likelihood estimator, one would take the likelihood function for the data and adjust the parameters to maximize this function. For a binomial distribution, we would depend on the known probability mass function to construct the likelihood function.

The process involves mathematical steps like taking derivatives and setting them to zero to find the maximum value. In the context of a binomially distributed variable, we use MLE to estimate the distribution's parameter, typically denoted by \(\theta\). From this parameter estimate, we can subsequently derive the estimators for the mean and variance of the distribution.

The Probability Mass Function (PMF) is a function that gives the probability that a discrete random variable is exactly equal to some value. For the binomial distribution, the PMF is denoted by \(p(x)\) and is calculated using the binomial formula which includes factorial functions, powers, and combinatorial coefficients represented by \(\binom{n}{x}\).

The PMF is fundamental when working with discrete data as it helps to characterize the distribution of the random variable. In practice, the PMF allows us to compute probabilities of various outcomes, which is an essential step in calculating the likelihood needed for MLE.

The binomial distribution is a discrete probability distribution that models the number of successes in a sequence of n independent experiments, each asking a yes-no question, and each with its own boolean-valued outcome: success (with probability \(\theta\)) or failure (with probability \(1-\theta\)).

Characteristics of the binomial distribution are governed by the parameters n (number of trials) and \(\theta\) (probability of success on an individual trial). The binomial distribution has a probability mass function, which is used to determine the likelihood of obtaining a specific number of successes across the trials. When using MLE for binomial distributions, we estimate \(\theta\), which can further lead to the estimators for the expected value and variance.

The bias of an estimator is a measure of how far the expected value of the estimator is from the true value of the parameter being estimated. An estimator is unbiased if its expected value equals the true parameter value. Bias is an important aspect because it informs us about the accuracy of the estimator in the long term.

In your textbook example, you'll find that while estimators obtained through MLE may be efficient, they can sometimes be biased. This is expressed mathematically through the Bias(\(\widehat{\mu}\)) and Bias(\(\widehat{\sigma^2}\)) formulas. It is essential to recognize that an estimator being unbiased is a desirable property, but not always possible to achieve. The bias of an estimator for variance in a binomial distribution does not tend to zero as n increases, which is a distinctive characteristic pointing out that despite having an efficient estimator, some level of bias persists.