Maximum likelihood, continuous random variables
Problem
We have a random sample where \(y_i\) follows an exponential distribution with parameter \(θ\) ,
\[y_i \sim exp(θ)\]
for \(i=1, \ldots ,n\) . The probability density is given by
\[f\left( y_i;θ \right)=θexp \left( -θy_i \right)\]
- What is \(L_i\left( θ \right)\) , the likelihood function for observation \(i\) ?
- What is \(l_i(θ)\) , the loglikelihood function for observation \(i\) ?
- Find the joint density \(f_J(y)\)
- Find the likelihood function \(L(θ)\)
- Find the loglikelihood function \(l(p)\) in two ways: by using \(l\left( θ \right)=log L(θ)\) and by using \(l\left( θ \right)=∑l_i\left( θ \right)\) . Use the symbol \(n_1=∑y_i\) .
- Find \(s_i(θ)\)
- Find \(s(θ)\) in two ways, by using \(s\left( θ \right)=dl/dθ\) and by using \(s\left( θ \right)=∑s_i\left( θ \right)\) .
- Find the stationary point of \(l(θ)\) , the solution to \(s\left( θ \right)=0\) (the maximum likelihood estimate)
- Find the second derivative \(d^2l_i/dθ^2\)
- Find \(I\left( θ \right)\) as \(-E\left( d^2l_i/dθ^2 \right)\)
- Given the following data, \(y_1=1.1, y_2=0.6,y_3=1.7,y_4=3.2\) find \({\hat{θ}}_{ML}\) and \(\hat{V}\) using j.
- Using the same data, find a 95% confidence interval for \(θ\) assuming that \(n=4\) is large enough for \(\sqrt{n}\left( {\hat{θ}}_{ML}-θ \right)\) to be approximately \(N(0,V)\) (yes, this is indeed a bit silly)
- In this problem, we clearly have \(I_H\left( θ \right)=I\left( θ \right)\) . Using the same data, find \(I_G\left( {\hat{θ}}_{ML} \right)\) .
Solution
a.
\[L_i\left( θ \right)=θexp \left( -θy_i \right)\]
b.
\[l_i\left( θ \right)=log θ-θy_i\]
c.
\[f_J\left( y \right)=\prod_{i=1}^{n}{ f\left( y_i;θ \right) }=\prod_{i=1}^{n}{ θexp \left( -θy_i \right) }=θ^nexp \left( -θ\sum_{i=1}^{n}{ y_i } \right)\]
d.
\[L\left( θ \right)=f_J\left( y \right)=θ^nexp \left( -θ\sum_{i=1}^{n}{ y_i } \right)\]
e.
First:
\[l\left( θ \right)=log L\left( θ \right)=nlog θ-θ\sum_{i=1}^{n}{ y_i }=nlog θ-θn_1\]
Second:
\[l\left( θ \right)=\sum_{i=1}^{n}{ l_i\left( θ \right) }=\sum_{i=1}^{n}{ \left( log θ-θy_i \right) }=nlog θ-θn_1\]
f.
\[s_i\left( θ \right)= \frac{∂l_i\left( θ \right)}{∂θ}= \frac{1}{θ}-y_i\]
g.
First:
\[s\left( θ \right)= \frac{dl}{dθ}= \frac{n}{θ}-n_1\]
Second:
\[s\left( θ \right)=\sum_{i=1}^{n}{ s_i\left( θ \right) }=\sum_{i=1}^{n}{ \left( \frac{1}{θ}-y_i \right) }= \frac{n}{θ}-n_1\]
h. \(s\left( θ \right)=0\) is \(n/θ-n_1=0\) with solution
\[{\hat{θ}}_{ML}= \frac{n}{n_1}= \frac{1}{\bar{y}}\]
i.
\[ \frac{d^2l_i}{dθ^2}= \frac{ds_i}{dθ}=- \frac{1}{θ^2}\]
j.
\[I\left( θ \right)=-E\left( \frac{∂^2l_i\left( θ \right)}{∂θ∂θ'} \right)=-E\left( - \frac{1}{θ^2} \right)= \frac{1}{θ^2}\]
k. \(\bar{y}=1.65\) and \({\hat{θ}}_{ML}=1/1.65=0.61\) . \(\hat{V}={0.61}^2=0.37\) .
l. Estimated variance of \({\hat{θ}}_{ML}\) is 0.37/4 =0.092 and \(SE\left( {\hat{θ}}_{ML} \right)=0.30\) so
\[θ=0.61±1.96⋅0.30\]
or
\[θ=0.61±0.59\]
m. \(I_H\left( {\hat{θ}}_{ML} \right)=I\left( {\hat{θ}}_{ML} \right)=1/{\hat{θ}}_{ML}^2=2.72.\)
\[I_G\left( {\hat{θ}}_{ML} \right)= \frac{1}{4}\sum_{i=1}^{4}{ {\left( 1.65-y_i \right)}^2 }=0.95\]