単回帰モデルの最良線形不偏推定量 α編

単回帰モデルの最良線形不偏推定量 $\alpha$編

単回帰モデル

$$\begin{array}{rcl}{y}_i& =& \alpha +\beta x_i+{ϵ}_i(i=1,\cdots ,n)\dots {ϵ}_i\stackrel{iid}{\sim }\mathrm{N}(0,{\sigma }^2)\end{array}$$

単回帰モデルに対する線形推定量の期待値

単回帰モデルの線形推定量に対して期待値を求めると，以下のように式展開される． $$\begin{array}{rcl}\mathrm{E}\left[\sum _{i=1}^n{c}_i{y}_i\right]& =& \mathrm{E}\left[\sum _{i=1}^n{c}_i(\alpha +\beta x_i+{ϵ}_i)\right]\cdots y_i=\alpha +\beta x_i+{ϵ}_i\\ & =& \mathrm{E}[\sum _{i=1}^n{c}_i\alpha +\sum _{i=1}^n{c}_i\beta x_i+\sum _{i=1}^n{c}_i{ϵ}_i]\cdots \sum (A+B)=\sum A+\sum B\\ & =& \mathrm{E}[\alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i+\sum _{i=1}^n{c}_i{ϵ}_i]\cdots \sum cA=c\sum A\\ & =& \alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i+\mathrm{E}\left[\sum _{i=1}^n{c}_i{ϵ}_i\right]\cdots \mathrm{E}[X+t]=\mathrm{E}\left[X\right]+t\\ & =& \alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i+\sum _{i=1}^n\mathrm{E}\left[c_i{ϵ}_i\right]\cdots \mathrm{E}\left[\sum _{i=1}^n{X}_i\right]=\mathrm{E}[X_1+\cdots +X_n]=\mathrm{E}\left[X_1\right]+\cdots +\mathrm{E}\left[X_n\right]=\sum _{i=1}^n\mathrm{E}\left[X_i\right]\\ & =& \alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i+\sum _{i=1}^n{c}_i\mathrm{E}\left[{ϵ}_i\right]\cdots \mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]\\ & =& \alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i+\sum _{i=1}^n{c}_i\cdot 0\cdots \mathrm{E}\left[{ϵ}_i\right]=0\\ & =& \alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i\end{array}$$

単回帰モデルに対する線形推定量の分散

$$\begin{array}{rcl}\mathrm{V}\left[\sum _{i=1}^n{c}_i{y}_i\right]& =& \sum _{i=1}^n\mathrm{V}\left[c_i{y}_i\right]\\ & =& \sum _{i=1}^n{c}_i^2\mathrm{V}\left[y_i\right]\\ & =& \sum _{i=1}^n{c}_i^2{\sigma }^2\\ & =& {\sigma }^2\sum _{i=1}^n{c}_i^2\end{array}$$

---

$\alpha$の線形推定量

単回帰モデルの推定量$\alpha$の線形推定量を$\tilde{\alpha }$とすると，この期待値は以下のように求められる． $$\begin{array}{rcl}\mathrm{E}\left[\tilde{\alpha }\right]& =& \mathrm{E}\left[\sum _{i=1}^n{c}_i{y}_i\right]\cdots \mathrm{線}\mathrm{形}\mathrm{推}\mathrm{定}\mathrm{量}\mathrm{と}\mathrm{す}\mathrm{る}\mathrm{仮}\mathrm{定}\mathrm{よ}\mathrm{り}\tilde{\alpha }=\sum _{i=1}^n{c}_i{y}_i\\ & =& \alpha \sum _{i=1}^n{c}_i+\beta \sum _{i=1}^n{c}_i{x}_i\end{array}$$ この$\tilde{\alpha }$が不偏推定量であるためには上記の$\tilde{\alpha }$の期待値が$\alpha$である必要がある.
よって式展開した結果の$\alpha ,\beta$の係数である$\sum _{i=1}^n{c}_i,\sum _{i=1}^n{c}_i{x}_i$が以下の条件を満たす必要があるが， 逆に条件を満たせば不偏推定量である． $$\begin{array}{r}\{\begin{array}{}\sum _{i=1}^n{c}_i& =& 1\\ \sum _{i=1}^n{c}_i{x}_i& =& 0\end{array}\end{array}$$ 不偏推定量となる線形推定量$\tilde{\alpha }$の中で分散${\sigma }^2\sum _{i=1}^n{c}_i^2$が最小となるものを 最良線形不偏推定量(best linear unbiased estimator; BLUE)と呼ぶ．ここでは${\hat{\alpha }}_{BLUE}$と表すこととする． $$\begin{array}{r}\{\begin{array}{}f(c_1,\cdots ,c_n)& =& {\sigma }^2\sum _{i=1}^n{c}_i^2& & \mathrm{最}\mathrm{小}& \mathrm{目}\mathrm{的}\mathrm{凾}\mathrm{数}\\ g(c_1,\cdots ,c_n)& =& \sum _{i=1}^n{c}_i-1& =& 0& \mathrm{制}\mathrm{約}\mathrm{条}\mathrm{件}\\ h(c_1,\cdots ,c_n)& =& \sum _{i=1}^n{c}_i{x}_i& =& 0& \mathrm{制}\mathrm{約}\mathrm{条}\mathrm{件}\end{array}\end{array}$$

---

${\hat{\alpha }}_{BLUE}$の最良線形不偏推定量を求める

ラグランジュの未定乗数法を用いる．束縛条件に係数をかけて目的凾数と線形結合した式を作成する． $$\begin{array}{rcl}L& =& L(c_1,\cdots ,c_n,\lambda ,\mu )\\ & =& f(c_1,\cdots ,c_n)+\lambda g(c_1,\cdots ,c_n)+\mu h(c_1,\cdots ,c_n)\\ & =& {\sigma }^2\left(\sum _{i=1}^n{c}_i^2\right)+\lambda (\sum _{i=1}^n{c}_i-1)+\mu \left(\sum _{i=1}^n{c}_i{x}_i\right)\end{array}$$ $L$を各変数$c_i$で微分したものを連立させる． $$\begin{array}{rclr}\frac{\partial L}{\partial c_i}& =& 0& (i=1,\cdots ,n)\\ 2{\sigma }^2{c}_i+\lambda +\mu x_i& =& 0& (i=1,\cdots ,n)& \cdots L\mathrm{を}{c}_i\mathrm{で}\mathrm{微}\mathrm{分}\mathrm{し}\mathrm{た}n\mathrm{個}\mathrm{の}\mathrm{等}\mathrm{式}\mathrm{が}\mathrm{で}\mathrm{き}\mathrm{る}\cdots \mathrm{式}A\end{array}$$ 連立させた式をまとめた式を一つ作る(その1)． $$\begin{array}{rclrc}2{\sigma }^2\sum _{i=1}^n{c}_i+\lambda \sum _{i=1}^{n}1+\mu \sum _{i=1}^n{x}_i& =& 0& & \cdots \mathrm{式}A\mathrm{を}i\mathrm{に}\mathrm{つ}\mathrm{い}\mathrm{て}\mathrm{和}\mathrm{を}\mathrm{と}\mathrm{っ}\mathrm{た}\\ 2{\sigma }^2\sum _{i=1}^n{c}_i+\lambda \cdot n+\mu n\overline{x}& =& 0& & \cdots \sum _{i=1}^{n}1=n,\sum _{i=1}^n{x}_i=n\overline{x}\\ 2{\sigma }^2\cdot 1+\lambda \cdot n+\mu n\overline{x}& =& 0& & \cdots \sum _{i=1}^n{c}_i=1(\mathrm{制}\mathrm{約}\mathrm{条}\mathrm{件})\\ 2{\sigma }^2+\lambda n+\mu n\overline{x}& =& 0\end{array}$$ 連立させた式をまとめた式を一つ作る(その2)． $$\begin{array}{rclrc}{c}_i(2{\sigma }^2{c}_i+\lambda +\mu x_i)& =& c_{i}0& (i=1,\cdots ,n)& \cdots \mathrm{式}A\mathrm{を}\mathrm{両}\mathrm{辺}{c}_i\mathrm{倍}\mathrm{し}\mathrm{た}\\ 2{\sigma }^2{c}_i^2+\lambda c_i+\mu c_i{x}_i& =& 0& (i=1,\cdots ,n)\\ 2{\sigma }^2\sum _{i=1}^n{c}_i^2+\lambda \sum _{i=1}^n{c}_i+\mu \sum _{i=1}^n{c}_i{x}_i& =& 0& & \cdots i\mathrm{に}\mathrm{つ}\mathrm{い}\mathrm{て}\mathrm{和}\mathrm{を}\mathrm{と}\mathrm{っ}\mathrm{た}\\ 2{\sigma }^2\sum _{i=1}^n{c}_i^2+\lambda \cdot 1+\mu \cdot 0& =& 0& & \cdots \sum _{i=1}^n{c}_i=1(\mathrm{制}\mathrm{約}\mathrm{条}\mathrm{件}),\sum _{i=1}^n{c}_i{x}_i=0(\mathrm{制}\mathrm{約}\mathrm{条}\mathrm{件})\\ 2{\sigma }^2\sum _{i=1}^n{c}_i^2+\lambda & =& 0\end{array}$$ 連立させた式をまとめた式を一つ作る(その3)． $$\begin{array}{rclrc}{x}_i(2{\sigma }^2{c}_i+\lambda +\mu x_i)& =& x_{i}0& (i=1,\cdots ,n)& \cdots \mathrm{式}A\mathrm{を}\mathrm{両}\mathrm{辺}{x}_i\mathrm{倍}\mathrm{し}\mathrm{た}\\ 2{\sigma }^2{c}_i{x}_i+\lambda x_i+\mu x_i^2& =& 0& (i=1,\cdots ,n)\\ 2{\sigma }^2\sum _{i=1}^n{c}_i{x}_i+\lambda \sum _{i=1}^n{x}_i+\mu \sum _{i=1}^n{x}_i^2& =& 0& & \cdots i\mathrm{に}\mathrm{つ}\mathrm{い}\mathrm{て}\mathrm{和}\mathrm{を}\mathrm{と}\mathrm{っ}\mathrm{た}\\ 2{\sigma }^2\sum _{i=1}^n{c}_i{x}_i+\lambda n\overline{x}+\mu \sum _{i=1}^n{x}_i^2& =& 0& & \cdots \sum _{i=1}^n{x}_i=n\overline{x}\\ 2{\sigma }^2\cdot 0+\lambda \cdot n\overline{x}+\mu \sum _{i=1}^n{x}_i^2& =& 0& & \cdots \sum _{i=1}^n{c}_i{x}_i=0(\mathrm{制}\mathrm{約}\mathrm{条}\mathrm{件})\\ \lambda n\overline{x}+\mu \sum _{i=1}^n{x}_i^2& =& 0\end{array}$$ まとめた式を改めて連立させる． $$\begin{array}{r}\{\begin{array}{}2{\sigma }^2+\lambda n+\mu n\overline{x}& =& 0\\ 2{\sigma }^2\sum _{i=1}^n{c}_i^2+\lambda & =& 0\\ \lambda n\overline{x}+\mu \sum _{i=1}^n{x}_i^2& =& 0\end{array}\end{array}$$

${\hat{\alpha }}_{BLUE}$の$\lambda ,\mu$を求める

改めて連立させた式を用いて定数$\lambda ,\mu$を求めていく． $$\begin{array}{rcl}2{\sigma }^2+\lambda n+\mu n\overline{x}& =& 0\\ \mu n\overline{x}& =& -2{\sigma }^2-\lambda n\\ \mu & =& \frac{-(2{\sigma }^2+\lambda n)}{n\overline{x}}\\ \lambda n\overline{x}+\mu \sum _{i=1}^n{x}_i^2& =& 0\\ \lambda n\overline{x}+\frac{-2{\sigma }^2-\lambda n}{n\overline{x}}\sum _{i=1}^n{x}_i^2& =& 0\\ n\overline{x}(\lambda n\overline{x}+\frac{-2{\sigma }^2-\lambda n}{n\overline{x}}\sum _{i=1}^n{x}_i^2)& =& n\overline{x}0\\ \lambda n^2{\overline{x}}^2+(-2{\sigma }^2-\lambda n)\sum _{i=1}^n{x}_i^2& =& 0\\ \lambda n^2{\overline{x}}^2-2{\sigma }^2\sum _{i=1}^n{x}_i^2-\lambda n\sum _{i=1}^n{x}_i^2& =& 0\\ \lambda n^2{\overline{x}}^2-\lambda n\sum _{i=1}^n{x}_i^2& =& 2{\sigma }^2\sum _{i=1}^n{x}_i^2\\ -n\lambda (\sum _{i=1}^n{x}_i^2-n{\overline{x}}^2)& =& 2{\sigma }^2\sum _{i=1}^n{x}_i^2\\ -n\lambda S_{xx}& =& 2{\sigma }^2\sum _{i=1}^n{x}_i^2\cdots S_{xx}=\sum _{i=1}^n{x}_i^2-n{\overline{x}}^2\\ \lambda & =& \frac{2{\sigma }^2\sum _{i=1}^n{x}_i^2}{-n{S}_{xx}}\\ & =& \frac{-2{\sigma }^2\sum _{i=1}^n{x}_i^2}{n{S}_{xx}}\\ \lambda n\overline{x}+\mu \sum _{i=1}^n{x}_i^2& =& 0\\ \mu \sum _{i=1}^n{x}_i^2& =& -\lambda n\overline{x}\\ \mu & =& \lambda \frac{-n\overline{x}}{\sum _{i=1}^n{x}_i^2}\\ & =& \frac{-2{\sigma }^2\sum _{i=1}^n{x}_i^2}{n{S}_{xx}}\frac{-n\overline{x}}{\sum _{i=1}^n{x}_i^2}\\ & =& \frac{2{\sigma }^2\overline{x}}{S_{xx}}\end{array}$$

${\hat{\alpha }}_{BLUE}$の$c_i$を求める

求めた定数$\lambda ,\mu$を用いて，最小となる変数$c_i$を求める． $$\begin{array}{rcl}2{\sigma }^2{c}_i+\lambda +\mu x_i& =& 0\\ c_i& =& -\frac{\lambda +\mu x_i}{2{\sigma }^2}\\ & =& -\frac{1}{2{\sigma }^2}(\frac{-2{\sigma }^2\sum _{i=1}^n{x}_i^2}{n{S}_{xx}}+\frac{2{\sigma }^2\overline{x}}{S_{xx}}{x}_i)\\ & =& -\frac{1}{2{\sigma }^2}(\frac{-2{\sigma }^2\sum _{i=1}^n{x}_i^2}{n{S}_{xx}}+\frac{2n{\sigma }^2\overline{x}}{n{S}_{xx}}{x}_i)\\ & =& -\frac{1}{2{\sigma }^2}\left(\frac{-2{\sigma }^2\sum _{i=1}^n{x}_i^2+2n{\sigma }^2\overline{x}{x}_i}{n{S}_{xx}}\right)\\ & =& -\frac{1}{2{\sigma }^2}\left(\frac{-2{\sigma }^2(\sum _{i=1}^n{x}_i^2-n\overline{x}{x}_i)}{n{S}_{xx}}\right)\\ & =& \frac{\sum _{i=1}^n{x}_i^2-n\overline{x}{x}_i}{n{S}_{xx}}\\ & =& \frac{A}{n}+\frac{B}{S_{xx}}\cdots \mathrm{部}\mathrm{分}\mathrm{分}\mathrm{数}\mathrm{分}\mathrm{解},A{S}_{xx}+Bn=\sum _{i=1}^n{x}_i^2-n\overline{x}{x}_i\\ & =& \frac{1}{n}+\frac{\overline{x}(\overline{x}-x_i)}{S_{xx}}\\ & & \cdots A{S}_{xx}+Bn=\sum _{i=1}^n{x}_i^2-n\overline{x}{x}_i\\ & & \cdots A(\sum _{i=1}^n{x}_i^2-n{\overline{x}}^2)+Bn=\sum _{i=1}^n{x}_i^2-n\overline{x}{x}_i\\ & & \cdots A\sum _{i=1}^n{x}_i^2-An{\overline{x}}^2+Bn=\sum _{i=1}^n{x}_i^2-n\overline{x}{x}_i\\ & & \cdots A\sum _{i=1}^n{x}_i^2=\sum _{i=1}^n{x}_i^2\overrightarrow{}A=1(\mathrm{係}\mathrm{数}\mathrm{比}\mathrm{較})\\ & & \cdots -An{\overline{x}}^2+Bn=-n\overline{x}{x}_i\overrightarrow{}B={\overline{x}}^2-\overline{x}{x}_i=\overline{x}(\overline{x}-x_i)(\mathrm{係}\mathrm{数}\mathrm{比}\mathrm{較})\\ & =& \frac{1}{n}-\frac{\overline{x}(x_i-\overline{x})}{S_{xx}}\end{array}$$ これは最小二乗推定量$\hat{\alpha }$と同じである． $$\begin{array}{rcl}{\hat{\alpha }}_{BLUE}& =& \sum _{i=1}^n{c}_i{y}_i\\ & =& \sum _{i=1}^n(\frac{1}{n}-\frac{\overline{x}(x_i-\overline{x})}{S_{xx}})y_i\cdots c_i=\frac{1}{n}-\frac{\overline{x}(x_i-\overline{x})}{S_{xx}}\\ & =& \sum _{i=1}^n\frac{1}{n}{y}_i-\sum _{i=1}^n\frac{\overline{x}(x_i-\overline{x})}{S_{xx}}{y}_i\cdots \sum (A_i+B_i)=\sum A_i+\sum B_i\\ & =& \frac{1}{n}\sum _{i=1}^n{y}_i-\frac{\overline{x}}{S_{xx}}\sum _{i=1}^n(x_i-\overline{x})y_i\cdots \sum c{A}_i=c\sum A_i\\ & =& \frac{1}{n}n\overline{y}-\frac{S_{xy}}{S_{xx}}\overline{x}\cdots \sum _{i=1}^n{y}_i=n\overline{y},\sum _{i=1}^n(x_i-\overline{x})y_i=S_{xy}\\ & =& \overline{y}-\hat{\beta }\overline{x}\cdots \hat{\beta }=\frac{S_{xy}}{S_{xx}}\\ & =& \hat{\alpha }\cdots \hat{\alpha }=\overline{y}-\hat{\beta }\overline{x}\end{array}$$

${\hat{\alpha }}_{BLUE}$の分散

${\hat{\alpha }}_{BLUE}$の分散${\sigma }^2\sum _{i=1}^n{c}_i^2$を最小とするための変数$c_i$を用いて以下のようになる． $$\begin{array}{rcl}2{\sigma }^2\sum _{i=1}^n{c}_i^2+\lambda & =& 0\\ {\sigma }^2\sum _{i=1}^n{c}_i^2& =& -\frac{1}{2}\lambda \\ & =& -\frac{1}{2}\frac{-2{\sigma }^2\sum _{i=1}^n{x}_i^2}{n{S}_{xx}}\\ & =& {\sigma }^2\frac{\sum _{i=1}^n{x}_i^2}{n{S}_{xx}}\\ & =& {\sigma }^2(\frac{A}{n}+\frac{B}{S_{xx}})\cdots \mathrm{部}\mathrm{分}\mathrm{分}\mathrm{数}\mathrm{分}\mathrm{解},A{S}_{xx}+Bn=\sum _{i=1}^n{x}_i^2\\ & =& {\sigma }^2(\frac{1}{n}+\frac{{\overline{x}}^2}{S_{xx}})\\ & & \cdots A{S}_{xx}+Bn=\sum _{i=1}^n{x}_i^2\\ & & \cdots A(\sum _{i=1}^n{x}_i^2-n{\overline{x}}^2)+Bn=\sum _{i=1}^n{x}_i^2\left(S_{xx}=\sum _{i=1}^n{x}_i^2-n{\overline{x}}^2\right)\\ & & \cdots A\sum _{i=1}^n{x}_i^2-An{\overline{x}}^2+Bn=\sum _{i=1}^n{x}_i^2\\ & & \cdots A\sum _{i=1}^n{x}_i^2=\sum _{i=1}^n{x}_i^2\overrightarrow{}A=1(\mathrm{係}\mathrm{数}\mathrm{比}\mathrm{較})\\ & & \cdots -An{\overline{x}}^2+Bn=0\overrightarrow{}B={\overline{x}}^2(\mathrm{係}\mathrm{数}\mathrm{比}\mathrm{較})\end{array}$$