91Ó°ÊÓ

An estimator is a rule for computing an estimate of a given quantity based on observable data: the rule (estimator), the quantity of interest (estimate), and the output (estimate) are all distinct.

a)

Let random variables \({{\rm{X}}_{\rm{1}}}{\rm{,}}{{\rm{X}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{X}}_{\rm{n}}}\) have joint pdf or pmb

\({\rm{f}}\left( {{{\rm{x}}_{\rm{1}}}{\rm{,}}{{\rm{x}}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{x}}_{\rm{n}}}{\rm{;}}{{\rm{\theta }}_{\rm{1}}}{\rm{,}}{{\rm{\theta }}_{\rm{2}}}{\rm{, \ldots ,}}{{\rm{\theta }}_{\rm{m}}}} \right){\rm{,}}\quad {\rm{n,m\^I N}}\)

Where the parameters \({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\)are unknown. When function \({\rm{f}}\)is a function of parameters\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\), it is called the

likelihood function

Values \({{\rm{\hat \theta }}_{\rm{i}}}\)that maximize the likelihood function are the maximum likelihood estimates (mle's), or equally values \({{\rm{\hat \theta }}_{\rm{i}}}\)for which

\(f\left( {{x_1},{x_2}, \ldots ,{x_n};{{\hat \theta }_1},{{\hat \theta }_2}, \ldots ,{{\hat \theta }_m}} \right)f\left( {{x_1},{x_2}, \ldots ,{x_n};{\theta _1},{\theta _2}, \ldots ,{\theta _m}} \right),\)

For every\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,m}}\). By substituting\({{\rm{X}}_{\rm{i}}}{\rm{ with }}{{\rm{x}}_{\rm{i}}}\), the

maximum likelihood estimators

are obtained.

First, notice that the random variable \({\rm{X}}\)has Binomial distribution with pdf given in the theorem below.

Theorem:

\({\rm{b(x;n,p) = }}\left\{ {\begin{array}{*{20}{l}}{\left( {\begin{array}{*{20}{l}}{\rm{n}}\\{\rm{x}}\end{array}} \right){{\rm{p}}^{\rm{x}}}{{{\rm{(1 - p)}}}^{{\rm{n - x}}}}}&{{\rm{,x = 0,1,2, \ldots ,n}}}\\{\rm{0}}&{{\rm{, otherwise }}}\end{array}} \right.\)

In order to obtain the maximum likelihood estimator, one needs to find \({\rm{p}}\)which maximizes pmf. To do that, look at natural logarithm of the pmf. By finding maximum of

\({\rm{ln}}\left( {\left( {\begin{array}{*{20}{l}}{\rm{n}}\\{\rm{x}}\end{array}} \right){{\rm{p}}^{\rm{x}}}{{{\rm{(1 - p)}}}^{{\rm{n - x}}}}} \right)\)

One also finds the maximum of\({{\rm{p}}^{\rm{x}}}{{\rm{(1 - p)}}^{{\rm{n - x}}}}\)because natural logarithm won't change the maximum value. To find maximum of\({\rm{lnb(x;n,p)}}\), first take the derivative and then set it to be equal to zero (classic method for finding maximum), and then solve for\({\rm{p}}\).

(a)

The derivative is:

\(\begin{aligned}\frac{{\rm{d}}}{{{\rm{dp}}}}\left( {{\rm{ln}}\left( {\left( {\begin{array}{*{20}{l}}{\rm{n}}\\{\rm{x}}\end{array}} \right){{\rm{p}}^{\rm{x}}}{{{\rm{(1 - p)}}}^{{\rm{n - x}}}}} \right)} \right)\\ &= \frac{{\rm{d}}}{{{\rm{dp}}}}\left( {{\rm{ln}}\left( {\begin{array}{*{20}{l}}{\rm{n}}\\{\rm{x}}\end{array}} \right){\rm{ + xlnp + (n - x)ln(1 - p)}}} \right)\\ & = 0 + x \times \frac{{\rm{1}}}{{\rm{p}}}{\rm{ + (n - x) \times }}\frac{{\rm{1}}}{{{\rm{1 - p}}}}{\rm{ \times ( - 1)}}\\ &= \frac{{\rm{x}}}{{\rm{p}}}{\rm{ - }}\frac{{{\rm{n - x}}}}{{{\rm{1 - p}}}}\end{aligned}\)

Set it to be equal to zero in order to find maximum

\(\frac{{\rm{x}}}{{\rm{p}}}{\rm{ - }}\frac{{{\rm{n - x}}}}{{{\rm{1 - p}}}}{\rm{ = 0}}\)

And now solve for \({\rm{p}}\)

\(\begin{aligned}\frac{{\rm{x}}}{{{\rm{\hat p}}}}{\rm{ - }}\frac{{{\rm{n - x}}}}{{{\rm{1 - \hat p}}}}\\ &= 0 \frac{{\rm{x}}}{{{\rm{\hat p}}}}\\ &= \frac{{{\rm{n - x}}}}{{{\rm{1 - \hat p}}}}\frac{{{\rm{1 - \hat p}}}}{{{\rm{\hat p}}}}\\ & = \frac{{{\rm{n - x}}}}{{\rm{x}}}\frac{{\rm{1}}}{{{\rm{\hat p}}}}{\rm{ - 1}}\\ &= \frac{{\rm{n}}}{{\rm{x}}}{\rm{ - 1}}\end{aligned}\)

Finally, the estimator is

\({\rm{\hat p = }}\frac{{\rm{X}}}{{\rm{n}}}{\rm{.}}\)

For \({\rm{n = 20}}\)and\({\rm{x = 3}}\), the estimate is\({\rm{\hat p = }}\frac{{\rm{3}}}{{{\rm{20}}}}{\rm{ = 0}}{\rm{.15}}\).

Thus, the estimated value is\({\rm{\hat p = }}\frac{{\rm{X}}}{{\rm{n}}}{\rm{;\hat p = 0}}{\rm{.15}}\).

b)

The estimator is unbiased if the expected value of the estimator is\({\rm{p}}\). The following holds

\(\begin{aligned} E(\hat p) &= E \left( {\frac{{\rm{X}}}{{\rm{n}}}} \right)\\ & = \frac{{\rm{1}}}{{\rm{n}}}{\rm{ \times E(X)}}\\\ &= \frac{{\rm{1}}}{{\rm{n}}}{\rm{ \times np}}\\ &= p \end{aligned}\)

(1): see the proposition below.

Proposition: For a binomial random variable \({\rm{X}}\)with parameters\({\rm{n,p}}\), and\({\rm{q = 1 - p}}\), the following is true

\(\begin{aligned}{l}E(X) &= np\\ V(X) &= np(1 - p) = npq \\{{\rm{\sigma }}_{\rm{X}}} &= \sqrt {{\rm{npq}}} \end{aligned}\)

Since, \({\rm{E(\hat p) = p}}\)

Therefore, theestimator is unbiased.

c)

The Invariance Principle:

Let \({{\rm{\hat \theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,n}}\)be maximum likelihood estimates of parameters\({{\rm{\theta }}_{\rm{i}}}{\rm{,i = 1,2, \ldots ,n}}\). The mle of any function of parameters \({{\rm{\theta }}_{\rm{i}}}\)is the function of the mle's\({{\rm{\hat \theta }}_{\rm{i}}}\).

The mle of function\({\rm{h(p) = (1 - p}}{{\rm{)}}^{\rm{5}}}\)is the function of \({\rm{\hat p}}\),

\(\begin{array}{l}{\rm{h(\hat p) = (1 - \hat p}}{{\rm{)}}^{\rm{5}}}\\{\rm{ = (1 - 0}}{\rm{.15}}{{\rm{)}}^{\rm{5}}}{\rm{ = 0}}{\rm{.4437}}{\rm{.}}\end{array}\)

Therefore, the required probability is\({\rm{h(\hat p) = 0}}{\rm{.4437}}\).