単回帰モデルの最尤推定量の期待値,分散,分布

単回帰モデルの最尤推定量の期待値,分散,分布

単回帰モデル

$$\begin{array}{rcl}{y}_i& =& \alpha +\beta x_i+{ϵ}_i(i=1,\cdots ,n)\\ & & {ϵ}_i\stackrel{iid}{\sim }N(0,{\sigma }^2)\cdots \mathrm{独}\mathrm{立}\mathrm{同}\mathrm{一}\mathrm{分}\mathrm{布}(independentandidenticallydistributed;IID,i.i.d.,iid)\end{array}$$ $\alpha ,\beta ,{\sigma }^2$の推定量を$\hat{\alpha },\hat{\beta },{\hat{\sigma }}^2$とし，$\hat{\alpha },\hat{\beta },{\hat{\sigma }}^2$の最尤推定量(maximum likelihood estimator)を${\hat{\alpha }}_{ML},{\hat{\beta }}_{ML},{\hat{\sigma }}_{ML}^2$とする．

${\hat{\beta }}_{ML}$の期待値

$$\begin{array}{rcl}\mathrm{E}\left[{\hat{\beta }}_{ML}\right]& =& \mathrm{E}\left[\hat{\beta }\right]\cdots {\hat{\beta }}_{ML}=\hat{\beta }\\ & =& \beta \cdots \mathrm{E}\left[\hat{\beta }\right]=\beta \\ & & \cdots \mathrm{よ}\mathrm{っ}\mathrm{て}{\hat{\beta }}_{ML}^2\mathrm{は}{\beta }^2\mathrm{の}\mathrm{不}\mathrm{偏}\mathrm{推}\mathrm{定}\mathrm{量}\mathrm{で}\mathrm{あ}\mathrm{る}.\end{array}$$

${\hat{\alpha }}_{ML}$の期待値

$$\begin{array}{rcl}\mathrm{E}\left[{\hat{\alpha }}_{ML}\right]& =& \mathrm{E}\left[\hat{\alpha }\right]\cdots {\hat{\alpha }}_{ML}=\hat{\alpha }\\ & =& \alpha \cdots \mathrm{E}\left[\hat{\alpha }\right]=\alpha \\ & & \cdots \mathrm{よ}\mathrm{っ}\mathrm{て}{\hat{\alpha }}_{ML}^2\mathrm{は}{\alpha }^2\mathrm{の}\mathrm{不}\mathrm{偏}\mathrm{推}\mathrm{定}\mathrm{量}\mathrm{で}\mathrm{あ}\mathrm{る}.\end{array}$$

${\hat{\beta }}_{ML}$の分散

$$\begin{array}{rcl}\mathrm{V}\left[{\hat{\beta }}_{ML}\right]& =& \mathrm{V}\left[\hat{\beta }\right]\\ & =& \frac{1}{S_{xx}}{\sigma }^2\cdots \mathrm{V}\left[\hat{\beta }\right]=\frac{1}{S_{xx}}{\sigma }^2\\ & & \cdots \overline{x}=\frac{1}{n}\sum _{i=0}^n{x}_i,S_{xx}=\sum _{i=0}^n{(x_i-\overline{x})}^2\end{array}$$

${\hat{\alpha }}_{ML}$の分散

$$\begin{array}{rcl}\mathrm{V}\left[{\hat{\alpha }}_{ML}\right]& =& \mathrm{V}\left[\hat{\alpha }\right]\\ & =& (\frac{1}{n}+\frac{{\overline{x}}^2}{S_{xx}}){\sigma }^2\cdots \mathrm{V}\left[\hat{\alpha }\right]=(\frac{1}{n}+\frac{{\overline{x}}^2}{S_{xx}}){\sigma }^2\end{array}$$

${\hat{\alpha }}_{ML},{\hat{\beta }}_{ML}$の分布

$$\begin{array}{rclrc}{\hat{\beta }}_{ML}& =& \hat{\beta }& \sim & \mathrm{N}(\beta ,\frac{1}{S_{xx}}{\sigma }^2)\cdots \hat{\beta }\sim \mathrm{N}(\beta ,\frac{1}{S_{xx}}{\sigma }^2)\\ {\hat{\alpha }}_{ML}& =& \hat{\alpha }& \sim & \mathrm{N}(\alpha ,(\frac{1}{n}+\frac{{\overline{x}}^2}{S_{xx}}){\sigma }^2)\cdots \hat{\alpha }\sim \mathrm{N}(\alpha ,(\frac{1}{n}+\frac{{\overline{x}}^2}{S_{xx}}){\sigma }^2)\end{array}$$

${\hat{\sigma }}_{ML}^2$の期待値

$$\begin{array}{rcl}\mathrm{E}\left[{\hat{\sigma }}_{ML}^2\right]& =& \mathrm{E}\left[\frac{n-2}{n}{s}^2\right]\cdots {\hat{\sigma }}_{ML}^2=\frac{n-2}{n}{s}^2,s^2=\frac{1}{(n-2)}\sum _{i=1}^n{e}_i^2,\sum _{i=1}^n{e}_i^2=\sum _{i=1}^n{(y_i-\widehat{y_i})}^2\\ & =& \frac{n-2}{n}\mathrm{E}\left[s^2\right]\\ & =& \frac{n-2}{n}{\sigma }^2\cdots \mathrm{E}\left[s^2\right]={\sigma }^2\\ & <& {\sigma }^2\cdots \mathrm{よ}\mathrm{っ}\mathrm{て}{\hat{\sigma }}_{ML}^2\mathrm{は}{\sigma }^2\mathrm{の}\mathrm{不}\mathrm{偏}\mathrm{推}\mathrm{定}\mathrm{量}\mathrm{で}\mathrm{は}\mathrm{な}\mathrm{い}.\end{array}$$