単回帰モデルの最尤推定量の期待値,分散,分布
単回帰モデル
$$
\begin{eqnarray}
y_i&=&\alpha+\beta x_i+\epsilon_i \;(i=1,\cdots,n)
\\&&\epsilon_i \overset{iid}{\sim} N(0,\sigma^2)\;\cdots\;独立同一分布(independent\;and\;identically\;distributed;\;IID,\;i.i.d.,\;iid)
\end{eqnarray}
$$
\(\alpha,\beta,\sigma^2\)の推定量を\(\hat{\alpha},\hat{\beta},\hat{\sigma}^2\)とし，\(\hat{\alpha},\hat{\beta},\hat{\sigma}^2\)の最尤推定量(maximum likelihood estimator)を\(\hat{\alpha}_{ML},\hat{\beta}_{ML},\hat{\sigma}^2_{ML}\)とする．
\(\hat{\beta}_{ML}\)の期待値
$$
\begin{eqnarray}
\mathrm{E}\left[\hat{\beta}_{ML}\right]&=&\mathrm{E}\left[\hat{\beta}\right]
\;\cdots\;\hat{\beta}_{ML}=\hat{\beta}
\\&=&\beta
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\beta}\right]=\beta}
\\&&\;\cdots\;よって\hat{\beta}_{ML}^2は\beta^2の不偏推定量で\mathbf{ある}.
\end{eqnarray}
$$
\(\hat{\alpha}_{ML}\)の期待値
$$
\begin{eqnarray}
\mathrm{E}\left[\hat{\alpha}_{ML}\right]&=&\mathrm{E}\left[\hat{\alpha}\right]
\;\cdots\;\hat{\alpha}_{ML}=\hat{\alpha}
\\&=&\alpha
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\alpha}\right]=\alpha}
\\&&\;\cdots\;よって\hat{\alpha}_{ML}^2は\alpha^2の不偏推定量で\mathbf{ある}.
\end{eqnarray}
$$
\(\hat{\beta}_{ML}\)の分散
$$
\begin{eqnarray}
\mathrm{V}\left[\hat{\beta}_{ML}\right]&=&\mathrm{V}\left[\hat{\beta}\right]
\\&=&\frac{1}{S_{xx}}\sigma^2
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\hat{\beta}\right]=\frac{1}{S_{xx}}\sigma^2}
\\&&\;\cdots\;\bar{x}=\frac{1}{n}\sum_{i=0}^{n}x_i,\;S_{xx}=\sum_{i=0}^{n}\left(x_i-\bar{x}\right)^2
\end{eqnarray}
$$
\(\hat{\alpha}_{ML}\)の分散
$$
\begin{eqnarray}
\mathrm{V}\left[\hat{\alpha}_{ML}\right]&=&\mathrm{V}\left[\hat{\alpha}\right]
\\&=&\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\hat{\alpha}\right]=\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2}
\end{eqnarray}
$$
\(\hat{\alpha}_{ML},\hat{\beta}_{ML}\)の分布
$$
\begin{eqnarray}
\hat{\beta}_{ML}&=&\hat{\beta}&\sim&\mathrm{N}\left(\beta,\;\frac{1}{S_{xx}}\sigma^2\right)
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post_29.html}{\hat{\beta}\sim\mathrm{N}\left(\beta,\;\frac{1}{S_{xx}}\sigma^2\right)}
\\\hat{\alpha}_{ML}&=&\hat{\alpha}&\sim&\mathrm{N}\left(\alpha,\;\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2\right)
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post_29.html}{\hat{\alpha}\sim\mathrm{N}\left(\alpha,\;\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2\right)}
\end{eqnarray}
$$
\(\hat{\sigma}^2_{ML}\)の期待値
$$
\begin{eqnarray}
\mathrm{E}\left[\hat{\sigma}_{ML}^2\right]&=&\mathrm{E}\left[\frac{n-2}{n}s^2\right]
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post_25.html}{\hat{\sigma}^2_{ML}=\frac{n-2}{n}s^2}
,\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{s^2=\frac{1}{\left(n-2\right)}\sum_{i=1}^{n} e_i^2}
,\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{\sum_{i=1}^{n}e_i^2=\sum_{i=1}^{n}\left(y_i-\hat{y_i}\right)^2}
\\&=&\frac{n-2}{n}\mathrm{E}\left[s^2\right]
\\&=&\frac{n-2}{n}\sigma^2
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{\mathrm{E}\left[s^2\right]=\sigma^2}
\\&\lt&\sigma^2
\;\cdots\;よって\hat{\sigma}_{ML}^2は\sigma^2の不偏推定量では\mathbf{ない}.
\end{eqnarray}
$$
0 件のコメント:
コメントを投稿