単回帰における残差平方和の期待値
誤差の平方和
$$ \begin{eqnarray} \sum_{i=1}^{n} \epsilon_i^2 &=&\sum_{i=1}^{n} \left(y_i-\alpha-\beta x_i\right)^2 \;\cdots\;y_i=\alpha+\beta x_i+\epsilon_i,\;誤差:\epsilon_i\overset{iid}{\sim} N(0,\sigma^2)\;互いに独立なN(0, \sigma^2)分布に従うと仮定する. \\&=&\sum_{i=1}^{n} \left( y_i -\alpha -\beta x_i \color{red}{ +\bar{y} -\hat{\alpha} -\hat{\beta}\bar{x} } \color{green}{ +\hat{\beta}x_i -\hat{\beta}x_i } \color{blue}{ +\beta\bar{x} -\beta\bar{x} } \right)^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{\bar{y}-\hat{\alpha}-\hat{\beta}\bar{x}=0} \\&=&\sum_{i=1}^{n} \left( y_i-\hat{\alpha}-\hat{\beta}x_i +\bar{y}-\alpha-\beta\bar{x} +\hat{\beta}x_i-\hat{\beta}\bar{x}-\beta x_i+\beta\bar{x} \right)^2 \\&=&\sum_{i=1}^{n} \left[ \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right) +\left(\bar{y}-\alpha-\beta\bar{x}\right) +\left(\hat{\beta}x_i-\hat{\beta}\bar{x}-\beta x_i+\beta\bar{x}\right) \right]^2 \\&=&\sum_{i=1}^{n} \left[ \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right) +\left(\bar{y}-\alpha-\beta\bar{x}\right) +\left\{\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right)\right\} \right]^2 \\&=&\sum_{i=1}^{n} \left[ \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)^2 +\left(\bar{y}-\alpha-\beta\bar{x}\right)^2 +\left\{\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right)\right\}^2 +2\left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)\left(\bar{y}-\alpha-\beta\bar{x}\right) +2\left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right) +2\left(\bar{y}-\alpha-\beta\bar{x}\right)\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right) \right] \\&&\;\cdots\;(A+B+C)^2=A^2+B^2+C^2+2AB+2AC+2BC \\&=&\sum_{i=1}^{n} \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)^2 +\sum_{i=1}^{n} \left(\bar{y}-\alpha-\beta\bar{x}\right)^2 +\sum_{i=1}^{n} \left(\hat{\beta}-\beta\right)^2\left(x_i-\bar{x}\right)^2 \\&&+2\sum_{i=1}^{n} \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)\left(\bar{y}-\alpha-\beta\bar{x}\right) +2\sum_{i=1}^{n} \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right) +2\sum_{i=1}^{n} \left(\bar{y}-\alpha-\beta\bar{x}\right)\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right) \\&=&\sum_{i=1}^{n} \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)^2 +\sum_{i=1}^{n} \left(\bar{y}-\alpha-\beta\bar{x}\right)^2 +\left(\hat{\beta}-\beta\right)^2 \sum_{i=1}^{n} \left(x_i-\bar{x}\right)^2 +2\cdot0+2\cdot0+2\cdot0 \\&&\;\cdots\;\sum_{i=1}^{n} \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)\left(\bar{y}-\alpha-\beta\bar{x}\right)=0\;(後述) \\&&\;\cdots\;\sum_{i=1}^{n} \left(y_i-\hat{\alpha}-\hat{\beta}x_i\right)\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right)=0\;(後述) \\&&\;\cdots\;\sum_{i=1}^{n} \left(\bar{y}-\alpha-\beta\bar{x}\right)\left(\hat{\beta}-\beta\right)\left(x_i-\bar{x}\right)＝0\;(後述) \\&=&\sum_{i=1}^{n}\left(y_i-\hat{y_i}\right)^2 +n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2 +\left(\hat{\beta}-\beta\right)^2S_{xx} \;\cdots\;\hat{y_i}=\hat{\alpha}+\hat{\beta}x_i,\; S_{xx}=\sum_{i=1}^{n} \left(x_i-\bar{x}\right)^2 \\&=&\sum_{i=1}^{n}e_i^2 +n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2 +\left(\hat{\beta}-\beta\right)^2S_{xx} \;\cdots\;e_i=y_i-\hat{y_i} \end{eqnarray} $$残差平方和\(S_e\)
前述の式で\(\sum_{i=1}^{n}\epsilon_i^2\)と\(\sum_{i=1}^{n}e_i^2\)を入れ替えて(互に移項して)以下の式を得る． $$ \begin{eqnarray} \sum_{i=1}^{n}e_i^2 &=&\sum_{i=1}^{n}\left(y_i-\hat{y_i}\right)^2 \;\cdots\;残差平方和\;(sum\;of\;squares\;of\;residuals;\;S_e) \\&=&\sum_{i=1}^{n} \epsilon_i^2 -n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2 -\left(\hat{\beta}-\beta\right)^2S_{xx} \end{eqnarray} $$残差平方和の期待値\(\mathrm{E}\left[S_e\right]\)
$$ \begin{eqnarray} \mathrm{E}\left[\sum_{i=1}^{n} e_i^2\right] &=&\mathrm{E}\left[\sum_{i=1}^{n} \epsilon_i^2 -n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2 -\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \\&=&\mathrm{E}\left[\sum_{i=1}^{n} \epsilon_i^2\right] -\mathrm{E}\left[n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[X+Y\right]=\mathrm{E}\left[X\right]+\mathrm{E}\left[Y\right]} \\&=&\sum_{i=1}^{n}\mathrm{E}\left[ \epsilon_i^2\right] -\mathrm{E}\left[n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \\&&\;\cdots\;\mathrm{E}\left[\sum_{i=1}^n X_i\right]=\mathrm{E}\left[X_1+\cdots+X_n\right]=\mathrm{E}\left[X_1\right]+\cdots+\mathrm{E}\left[X_n\right]=\sum_{i=1}^n\mathrm{E}\left[X_i\right] \\&=&\sum_{i=1}^{n}\left(\mathrm{V}\left[ \epsilon_i \right]+\mathrm{E}\left[ \epsilon_i \right]^2\right) -\mathrm{E}\left[n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \\&&\;\cdots\;\mathrm{V}\left[ X \right]=\mathrm{E}\left[ \left(X-\mathrm{E}\left[ X \right]\right)^2 \right]=\mathrm{E}\left[ X^2 \right]-\mathrm{E}\left[ X \right]^2,\;\mathrm{E}\left[ X^2 \right]=\mathrm{V}\left[ X \right]+\mathrm{E}\left[ X \right]^2 \\&=&\sum_{i=1}^{n}\left(\sigma^2+0^2\right) -\mathrm{E}\left[n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \;\cdots\;\epsilon_i\overset{iid}{\sim} N(0,\sigma^2),\;\mathrm{E}\left[ \epsilon_i \right]=0,\;\mathrm{V}\left[ \epsilon_i \right]=\sigma^2 \\&=&\sum_{i=1}^{n}\sigma^2 -\mathrm{E}\left[n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \\&=&\sigma^2\sum_{i=1}^{n}1 -\mathrm{E}\left[n\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2S_{xx}\right] \\&=&n\sigma^2 -n\mathrm{E}\left[\left(\bar{y}-\alpha-\beta\bar{x}\right)^2\right] -S_{xx}\mathrm{E}\left[\left(\hat{\beta}-\beta\right)^2\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]} \\&=&n\sigma^2 -n\mathrm{V}\left[\bar{y}\right] -S_{xx}\mathrm{V}\left[\hat{\beta}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[X+t\right]=\mathrm{V}\left[X\right]\;(t:定数)} \\&=&n\sigma^2 -n\frac{\sigma^2}{n} -S_{xx}\mathrm{V}\left[\frac{S_{xy}}{S_{xx}}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/specimen-random-variable_3.html}{\mathrm{V}\left[\bar{y}\right]=\frac{\sigma^2}{n}} ,\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{\hat{\beta}=\frac{S_{xy}}{S_{xx}}} \\&=&n\sigma^2 -n\frac{\sigma^2}{n} -S_{xx}\frac{1}{S_{xx}^2}\mathrm{V}\left[S_{xy}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]\;(c:定数)} \\&=&n\sigma^2 -\sigma^2 -\frac{1}{S_{xx}}\sigma^2S_{xx} \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[S_{xy}\right]=\sigma^2S_{xx}} \\&=&n\sigma^2 -\sigma^2 -\sigma^2 \\&=&\left(n-2\right)\sigma^2 \;\cdots\;残差平方和(S_e)の期待値 \end{eqnarray} $$ よって\((n-2)\)で残差平方和を割っておけば\(\sigma^2\)が得られ不偏推定量となる． $$ \begin{eqnarray} \mathrm{E}\left[\sum_{i=1}^{n} e_i^2\right]&=&\left(n-2\right)\sigma^2 \\\frac{1}{n-2}\mathrm{E}\left[\sum_{i=1}^{n} e_i^2\right]&=&\sigma^2 \\\mathrm{E}\left[\frac{1}{\left(n-2\right)}\sum_{i=1}^{n} e_i^2\right]&=&\sigma^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{c\mathrm{E}\left[X\right]=\mathrm{E}\left[cX\right]} \\\mathrm{E}\left[s^2\right]&=&\sigma^2 \;\cdots\;s^2=\frac{1}{\left(n-2\right)}\sum_{i=1}^{n} e_i^2は\sigma^2の不偏推定量 \end{eqnarray} $$標準化残差\(e_{is}\)とその分布
$$ \begin{eqnarray} \\e_{is}&\equiv&\frac{e_i}{s} \;\cdots\;標準化残差\;(standardized\;residuals),\;s\equiv\sqrt{\frac{1}{\left(n-2\right)}\sum_{i=1}^{n} e_i^2} \\e_i&=&y_i-\hat{y_i}=(\alpha+\beta x_i+\epsilon_i)-(\hat{\alpha}+\hat{\beta}x_i) \\&=&(\alpha-\hat{\alpha})+(\beta-\hat{\beta})x_i+\epsilon_i \end{eqnarray} $$ \(n\)が十分大きい時\(\hat{\alpha},\hat{\beta}\)は\(\alpha,\beta\)に確率収束する( \(\hat{\alpha},\hat{\beta}\)は不偏推定量(期待値は\(\alpha,\beta\)) )．よって $$ \begin{eqnarray} e_i&\sim&\epsilon_i \;\cdots\;(\alpha-\hat{\alpha}),(\beta-\hat{\beta})はほぼ0になり第1,2項は無視できるようになる． \end{eqnarray} $$ となる．また， $$ \begin{eqnarray} e_{is}&=&\frac{e_i}{s} \end{eqnarray} $$ は， $$ \begin{eqnarray} \epsilon_{is}&\equiv&\frac{\epsilon_i}{\sigma} \;\cdots\;nが十分大きい時,\;sは\sigmaに確率収束する(s^2は不偏推定量(期待値は\sigma^2))． \end{eqnarray} $$ と同じ分布に従うとみなせる．\(\epsilon_{i}\sim N(0,\sigma^2)\)に従うので\(\epsilon_{is}\)は\(N(0, 1^2)\)に従うと考えられる．
0 件のコメント:
コメントを投稿