式展開

回帰による変動の平方和と全平方和の比率

また全平方和$S_{T}$のうち回帰による変動の平方和$S_{R}$の比率を寄与率，あるいは決定係数$R^2$という． $$\begin{eqnarray} R^2&=&\frac{S_{R}}{S_{T}}\\ &=&\frac{S_{R}}{S_{yy}}\;\dots\;S_{T}=S_{yy}\\ &=&\frac{\frac{S_{xy}^2}{S_{xx}}}{S_{yy}}\;\dots\;S_{R}=\frac{S_{xy}^2}{S_{xx}}\\ &=&\frac{S_{xy}^2}{S_{xx}S_{yy}}\\ &=&\left(\frac{S_{xy}}{\sqrt{S_{xx}}\sqrt{S_{yy}}}\right)^2\\ &=&\left(\frac{\frac{S_{xy}}{n-1}}{\sqrt{\frac{S_{xx}}{n-1}}\sqrt{\frac{S_{yy}}{n-1}}}\right)^2\\ &=&\frac{Cov[X, Y]^2}{V[X]V[Y]}\;\dots\; Cov\left[ X,Y \right] = E\left[ \left\{ X-E \left[ X \right] \right\} \left\{ Y-E \left[ Y \right] \right\} \right],\;V\left[ X\right] = E\left[ \left\{ X-E \left[ X \right] \right\}^2\right]\\ &=&\rho^2\;\dots\;\frac{Cov[X, Y]}{\sqrt{V[X]}\sqrt{V[Y]}}=\rho (相関係数)\\ R^2=\frac{S_{R}}{S_{T}}&=& \frac{S_{T}-S{e}}{S_{T}}=1-\frac{S_{e}}{S_{T}}\;\dots\;S_{R}=S_{T}-S{e}\\ &=& \frac{S_{T}-S{e}}{S_{yy}}=1-\frac{S_{e}}{S_{yy}}\\ \end{eqnarray}$$

残差平方和とデータの各平方和

$$\begin{eqnarray} S_e&=&\sum_{i=1}^n e_i^2\\ &=&\sum_{i=1}^n \left(y_i - \hat{y_i}\right)^2\;\dots\;\hat{y_i}=\hat{\alpha}+\hat{\beta} x_i\\ &=&\left(\sum_{i=1}^n y_i^2\right) - 2n\bar{y}\hat{\alpha} - 2\left(\sum_{i=1}^n x_i y_i\right)\hat{\beta} + n\hat{\alpha}^2 + 2n\bar{x}\hat{\alpha}\hat{\beta} + \left(\sum_{i=1}^n x_i^2\right)\hat{\beta}^2 \;\dots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{残差平方和の展開した式}\\ &=&\left(\sum_{i=1}^n y_i^2\right) - 2n\bar{y}\left(\bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x}\right) - 2\left(\sum_{i=1}^n x_i y_i\right)\frac{S_{xy}}{S_{xx}} + n\left(\bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x}\right)^2 + 2n\bar{x}\left(\bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x}\right)\frac{S_{xy}}{S_{xx}} + \left(\sum_{i=1}^n x_i^2\right)\left(\frac{S_{xy}}{S_{xx}}\right)^2 \;\dots\; \hat{\alpha}=\bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x},\;\hat{\beta}=\frac{S_{xy}}{S_{xx}}\\ &=&\left(\sum_{i=1}^n y_i^2\right) - 2n\bar{y}^2 + 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} - 2\frac{S_{xy}}{S_{xx}}\left(\sum_{i=1}^n x_i y_i\right) + n\bar{y}^2 - 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} + n\left(\frac{S_{xy}}{S_{xx}}\right)^2\bar{x}^2 + 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} - 2n\left(\frac{S_{xy}}{S_{xx}}\right)^2\bar{x}^2 + \left(\sum_{i=1}^n x_i^2\right)\left(\frac{S_{xy}}{S_{xx}}\right)^2\\ &=&\left\{ \left(\sum_{i=1}^n y_i^2\right) - 2n\bar{y}^2 + n\bar{y}^2 \right\} +\left\{ 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} - 2\frac{S_{xy}}{S_{xx}}\left(\sum_{i=1}^n x_i y_i\right) - 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} + 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} \right\} +\left\{ n\left(\frac{S_{xy}}{S_{xx}}\right)^2\bar{x}^2 - 2n\left(\frac{S_{xy}}{S_{xx}}\right)^2\bar{x}^2 + \left(\sum_{i=1}^n x_i^2\right)\left(\frac{S_{xy}}{S_{xx}}\right)^2 \right\}\\ &=&\left\{ \left(\sum_{i=1}^n y_i^2\right) - n\bar{y}^2 \right\} +\left\{ 2n\frac{S_{xy}}{S_{xx}}\bar{x}\bar{y} - 2\frac{S_{xy}}{S_{xx}}\left(\sum_{i=1}^n x_i y_i\right) \right\} +\left\{ - n\left(\frac{S_{xy}}{S_{xx}}\right)^2\bar{x}^2 + \left(\sum_{i=1}^n x_i^2\right)\left(\frac{S_{xy}}{S_{xx}}\right)^2 \right\}\\ &=&\left\{ \left(\sum_{i=1}^n y_i^2\right) - n\bar{y}^2 \right\} -2\frac{S_{xy}}{S_{xx}}\left\{ \left(\sum_{i=1}^n x_i y_i\right) - n\bar{x}\bar{y} \right\} +\left(\frac{S_{xy}}{S_{xx}}\right)^2\left\{ \left(\sum_{i=1}^n x_i^2\right) - n\bar{x}^2 \right\} \;\dots\;\sum_{i=1}^n y_i^2-n\bar{y}^2=S_{yy},\;\sum_{i=1}^n x_i^2-n\bar{x}^2=S_{xx},\;\sum_{i=1}^n x_i y_i-n\bar{x}\bar{y}=S_{xy}\\ &=&S_{yy} -2\frac{S_{xy}}{S_{xx}}S_{xy} +\left(\frac{S_{xy}}{S_{xx}}\right)^2S_{xx}\\ &=&S_{yy} -2\frac{S_{xy}^2}{S_{xx}} +\frac{S_{xy}^2}{S_{xx}}\\ &=&S_{yy} -\frac{S_{xy}^2}{S_{xx}}\\ \end{eqnarray}$$ よって，残差平方和$S_{e}$はデータの各平方和($S_{xx}, S_{yy}, S_{xy}$)より求めることができる．

全平方和の分解

以下のような和を考える．第一項は回帰直線から推定される値$\hat{y_i}$と目的変数の平均値$\bar{y}$との差の平方和であり，第二項は残差平方和である． $$\begin{eqnarray} &&\sum_{i=0}^n\left(\hat{y_i}-\bar{y}\right)^2+\sum_{i=0}^n\left(y_i-\hat{y_i}\right)^2\\ &=&\sum_{i=0}^n\left(\hat{y_i}^2-2\hat{y_i}\bar{y}+\bar{y}^2\right)+\sum_{i=0}^n\left(y_i^2-2y_i\hat{y_i}+\hat{y_i}^2\right)\\ &=&\sum_{i=0}^n\left\{ \left(\alpha+\beta x_i\right)^2-2\left(\alpha+\beta x_i\right)\bar{y}+\bar{y}^2 \right\} +\sum_{i=0}^n\left\{ y_i^2-2y_i\left(\alpha+\beta x_i\right)+\left(\alpha+\beta x_i\right)^2 \right\}\\ &=&\sum_{i=0}^n\left(\alpha^2+2\alpha\beta x_i+\beta^2x_i^2-2\alpha\bar{y}-2\beta\bar{y}x_i+\bar{y}^2\right) +\sum_{i=0}^n\left(y_i^2-2\alpha y_i-2\beta x_i y_i+\alpha^2+2\alpha\beta x_i+\beta^2 x_i^2\right)\\ &=&\sum_{i=0}^n\left\{ \left(\bar{y}-\beta\bar{x}\right)^2 +2\left(\bar{y}-\beta\bar{x}\right)\beta x_i +\beta^2x_i^2 -2\left(\bar{y}-\beta\bar{x}\right)\bar{y} -2\beta\bar{y}x_i+\bar{y}^2 \right\} +\sum_{i=0}^n\left\{ y_i^2-2\left(\bar{y}-\beta\bar{x}\right) y_i-2\beta x_i y_i+\left(\bar{y}-\beta\bar{x}\right)^2+2\left(\bar{y}-\beta\bar{x}\right)\beta x_i+\beta^2 x_i^2 \right\}\\ &=&\sum_{i=0}^n\left( \bar{y}^2-2\beta\bar{x}\bar{y}+\beta^2\bar{x}^2 +2\beta \bar{y}x_i -2\beta^2\bar{x} x_i +\beta^2x_i^2 -2\bar{y}^2 +2\beta\bar{x}\bar{y} -2\beta\bar{y}x_i+\bar{y}^2 \right) +\sum_{i=0}^n\left( y_i^2 -2\bar{y}y_i+2\beta\bar{x}y_i -2\beta x_i y_i +\bar{y}^2-2\beta\bar{x}\bar{y}+\beta^2\bar{x}^2 +2\beta\bar{y} x_i-2\beta^2\bar{x} x_i +\beta^2 x_i^2 \right)\\ &=&\bar{y}^2\left(\sum_{i=0}^n 1\right) -2\beta\bar{x}\bar{y}\left(\sum_{i=0}^n 1\right) +\beta^2\bar{x}^2\left(\sum_{i=0}^n 1\right) +2\beta \bar{y}\left(\sum_{i=0}^n x_i\right) -2\beta^2\bar{x}\left(\sum_{i=0}^n x_i\right) +\beta^2 \left(\sum_{i=0}^n x_i^2\right) -2\bar{y}^2 \left(\sum_{i=0}^n 1\right) +2\beta\bar{x}\bar{y} \left(\sum_{i=0}^n 1\right) -2\beta\bar{y} \left(\sum_{i=0}^n x_i\right) +\bar{y}^2 \left(\sum_{i=0}^n 1\right)\\ &&+\left(\sum_{i=0}^n y_i^2 \right) -2\bar{y}\left(\sum_{i=0}^n y_i\right) +2\beta\bar{x}\left(\sum_{i=0}^n y_i\right) -2\beta \left(\sum_{i=0}^n x_i y_i\right) +\bar{y}^2\left(\sum_{i=0}^n 1\right) -2\beta\bar{x}\bar{y}\left(\sum_{i=0}^n 1\right) +\beta^2\bar{x}^2 \left(\sum_{i=0}^n 1\right) +2\beta\bar{y} \left(\sum_{i=0}^n x_i\right) -2\beta^2\bar{x} \left(\sum_{i=0}^n x_i\right) +\beta^2 \left(\sum_{i=0}^n x_i^2 \right)\\ &=&n\bar{y}^2 -2n\beta\bar{x}\bar{y} +n\beta^2\bar{x}^2 +2n\beta\bar{x}\bar{y} -2n\beta^2\bar{x}^2 +\beta^2 \left(\sum_{i=0}^n x_i^2\right) -2n\bar{y}^2 +2n\beta\bar{x}\bar{y} -2n\beta\bar{x}\bar{y} +n\bar{y}^2\\ &&+\left(\sum_{i=0}^n y_i^2 \right) -2n\bar{y}^2 +2n\beta\bar{x}\bar{y} -2\beta \left(\sum_{i=0}^n x_i y_i\right) +n\bar{y}^2 -2n\beta\bar{x}\bar{y} +n\beta^2\bar{x}^2 +2n\beta\bar{x}\bar{y} -2n\beta^2\bar{x}^2 +\beta^2 \left(\sum_{i=0}^n x_i^2 \right)\\ &=&\left\{ \left(\sum_{i=0}^n y_i^2 \right) +n\bar{y}^2 -2n\bar{y}^2 +n\bar{y}^2 -2n\bar{y}^2 +n\bar{y}^2 \right\} +\left\{ 2\beta^2 \left(\sum_{i=0}^n x_i^2\right) +n\beta^2\bar{x}^2 -2n\beta^2\bar{x}^2 +n\beta^2\bar{x}^2 -2n\beta^2\bar{x}^2 \right\} +\left\{ -2\beta \left(\sum_{i=0}^n x_i y_i\right) -2n\beta\bar{x}\bar{y} +2n\beta\bar{x}\bar{y} +2n\beta\bar{x}\bar{y} -2n\beta\bar{x}\bar{y} +2n\beta\bar{x}\bar{y} -2n\beta\bar{x}\bar{y} +2n\beta\bar{x}\bar{y} \right\}\\ &=&\left\{ \left(\sum_{i=0}^n y_i^2 \right) -n\bar{y}^2 \right\} +2\beta^2\left\{ \left(\sum_{i=0}^n x_i^2\right) -n\bar{x}^2 \right\} -2\beta \left\{ \left(\sum_{i=0}^n x_i y_i\right) -n\bar{x}\bar{y} \right\}\\ &=&S_{yy}+2\beta^2 S_{xx}-2\beta S_{xy}\\ &=&S_{yy}+2\beta \left(\beta S_{xx} - S_{xy}\right)\\ &=&S_{yy}+2\left(\frac{S_{xy}}{S_{xx}}\right) \left\{\left(\frac{S_{xy}}{S_{xx}}\right) S_{xx} - S_{xy}\right\}\\ &=&S_{yy}+2\left(\frac{S_{xy}}{S_{xx}}\right) \left(S_{xy} - S_{xy}\right)\\ &=&S_{yy} = \sum_{i=0}^n \left(y_i - \bar{y}\right)^2\\ &=&S_{T} \end{eqnarray}$$ よって，全平方和$S_{T}$である$S_{yy}$は回帰直線から推定される値$\hat{y}$と目的変数の平均値$\bar{y}$との差の平方和と残差平方和で表せられる．
この回帰直線から推定される値$\hat{y}$と目的変数の平均値$\bar{y}$との差の平方和を，回帰による変動の平方和$S_{R}$と呼ぶ． $$\begin{eqnarray} \sum_{i=0}^n \left(y_i - \bar{y}\right)^2&=&\sum_{i=0}^n\left(\hat{y_i}-\bar{y}\right)^2+\sum_{i=0}^n\left(y_i-\hat{y_i}\right)^2\\ S_{T}&=&S_{R}+S_{e}\\ S_{yy}&=&S_{R}+S_{e}\\ S_{yy}&=&S_{R}+\left(S_{yy}-\frac{S_{xy}^2}{S_{xx}}\right)\\ S_{R}&=&\frac{S_{xy}^2}{S_{xx}}\\ \end{eqnarray}$$ 回帰による変動の平方和$S_{R}$もデータの各平方和($S_{xx}, S_{yy}$)より求めることができる．