全平方和の分解

全平方和の分解

以下のような和を考える．第一項は回帰直線から推定される値$\widehat{y_i}$と目的変数の平均値$\overline{y}$との差の 平方和であり，第二項は残差平方和である． $$\begin{array}{rcl}& & \sum _{i=0}^n{(\widehat{y_i}-\overline{y})}^2+\sum _{i=0}^n{(y_i-\widehat{y_i})}^2\\ & =& \sum _{i=0}^n({\widehat{y_i}}^2-2\widehat{y_i}\overline{y}+{\overline{y}}^2)+\sum _{i=0}^n(y_i^2-2y_i\widehat{y_i}+{\widehat{y_i}}^2)\\ & =& \sum _{i=0}^n\{{(\alpha +\beta x_i)}^2-2(\alpha +\beta x_i)\overline{y}+{\overline{y}}^2\}+\sum _{i=0}^n\{y_i^2-2y_i(\alpha +\beta x_i)+{(\alpha +\beta x_i)}^2\}\\ & =& \sum _{i=0}^n({\alpha }^2+2\alpha \beta x_i+{\beta }^2{x}_i^2-2\alpha \overline{y}-2\beta \overline{y}{x}_i+{\overline{y}}^2)+\sum _{i=0}^n(y_i^2-2\alpha y_i-2\beta x_i{y}_i+{\alpha }^2+2\alpha \beta x_i+{\beta }^2{x}_i^2)\\ & =& \sum _{i=0}^n\{{(\overline{y}-\beta \overline{x})}^2+2(\overline{y}-\beta \overline{x})\beta x_i+{\beta }^2{x}_i^2-2(\overline{y}-\beta \overline{x})\overline{y}-2\beta \overline{y}{x}_i+{\overline{y}}^2\}+\sum _{i=0}^n\{y_i^2-2(\overline{y}-\beta \overline{x})y_i-2\beta x_i{y}_i+{(\overline{y}-\beta \overline{x})}^2+2(\overline{y}-\beta \overline{x})\beta x_i+{\beta }^2{x}_i^2\}\\ & =& \sum _{i=0}^n({\overline{y}}^2-2\beta \overline{x}\overline{y}+{\beta }^2{\overline{x}}^2+2\beta \overline{y}{x}_i-2{\beta }^2\overline{x}{x}_i+{\beta }^2{x}_i^2-2{\overline{y}}^2+2\beta \overline{x}\overline{y}-2\beta \overline{y}{x}_i+{\overline{y}}^2)+\sum _{i=0}^n(y_i^2-2\overline{y}{y}_i+2\beta \overline{x}{y}_i-2\beta x_i{y}_i+{\overline{y}}^2-2\beta \overline{x}\overline{y}+{\beta }^2{\overline{x}}^2+2\beta \overline{y}{x}_i-2{\beta }^2\overline{x}{x}_i+{\beta }^2{x}_i^2)\\ & =& {\overline{y}}^2\left(\sum _{i=0}^{n}1\right)-2\beta \overline{x}\overline{y}\left(\sum _{i=0}^{n}1\right)+{\beta }^2{\overline{x}}^2\left(\sum _{i=0}^{n}1\right)+2\beta \overline{y}\left(\sum _{i=0}^n{x}_i\right)-2{\beta }^2\overline{x}\left(\sum _{i=0}^n{x}_i\right)+{\beta }^2\left(\sum _{i=0}^n{x}_i^2\right)-2{\overline{y}}^2\left(\sum _{i=0}^{n}1\right)+2\beta \overline{x}\overline{y}\left(\sum _{i=0}^{n}1\right)-2\beta \overline{y}\left(\sum _{i=0}^n{x}_i\right)+{\overline{y}}^2\left(\sum _{i=0}^{n}1\right)\\ & & +\left(\sum _{i=0}^n{y}_i^2\right)-2\overline{y}\left(\sum _{i=0}^n{y}_i\right)+2\beta \overline{x}\left(\sum _{i=0}^n{y}_i\right)-2\beta \left(\sum _{i=0}^n{x}_i{y}_i\right)+{\overline{y}}^2\left(\sum _{i=0}^{n}1\right)-2\beta \overline{x}\overline{y}\left(\sum _{i=0}^{n}1\right)+{\beta }^2{\overline{x}}^2\left(\sum _{i=0}^{n}1\right)+2\beta \overline{y}\left(\sum _{i=0}^n{x}_i\right)-2{\beta }^2\overline{x}\left(\sum _{i=0}^n{x}_i\right)+{\beta }^2\left(\sum _{i=0}^n{x}_i^2\right)\\ & =& n{\overline{y}}^2-2n\beta \overline{x}\overline{y}+n{\beta }^2{\overline{x}}^2+2n\beta \overline{x}\overline{y}-2n{\beta }^2{\overline{x}}^2+{\beta }^2\left(\sum _{i=0}^n{x}_i^2\right)-2n{\overline{y}}^2+2n\beta \overline{x}\overline{y}-2n\beta \overline{x}\overline{y}+n{\overline{y}}^2\\ & & +\left(\sum _{i=0}^n{y}_i^2\right)-2n{\overline{y}}^2+2n\beta \overline{x}\overline{y}-2\beta \left(\sum _{i=0}^n{x}_i{y}_i\right)+n{\overline{y}}^2-2n\beta \overline{x}\overline{y}+n{\beta }^2{\overline{x}}^2+2n\beta \overline{x}\overline{y}-2n{\beta }^2{\overline{x}}^2+{\beta }^2\left(\sum _{i=0}^n{x}_i^2\right)\\ & =& \{\left(\sum _{i=0}^n{y}_i^2\right)+n{\overline{y}}^2-2n{\overline{y}}^2+n{\overline{y}}^2-2n{\overline{y}}^2+n{\overline{y}}^2\}+\{2{\beta }^2\left(\sum _{i=0}^n{x}_i^2\right)+n{\beta }^2{\overline{x}}^2-2n{\beta }^2{\overline{x}}^2+n{\beta }^2{\overline{x}}^2-2n{\beta }^2{\overline{x}}^2\}+\{-2\beta \left(\sum _{i=0}^n{x}_i{y}_i\right)-2n\beta \overline{x}\overline{y}+2n\beta \overline{x}\overline{y}+2n\beta \overline{x}\overline{y}-2n\beta \overline{x}\overline{y}+2n\beta \overline{x}\overline{y}-2n\beta \overline{x}\overline{y}+2n\beta \overline{x}\overline{y}\}\\ & =& \{\left(\sum _{i=0}^n{y}_i^2\right)-n{\overline{y}}^2\}+2{\beta }^2\{\left(\sum _{i=0}^n{x}_i^2\right)-n{\overline{x}}^2\}-2\beta \{\left(\sum _{i=0}^n{x}_i{y}_i\right)-n\overline{x}\overline{y}\}\\ & =& S_{yy}+2{\beta }^2{S}_{xx}-2\beta S_{xy}\\ & =& S_{yy}+2\beta (\beta S_{xx}-S_{xy})\\ & =& S_{yy}+2\left(\frac{S_{xy}}{S_{xx}}\right)\{\left(\frac{S_{xy}}{S_{xx}}\right)S_{xx}-S_{xy}\}\\ & =& S_{yy}+2\left(\frac{S_{xy}}{S_{xx}}\right)(S_{xy}-S_{xy})\\ & =& S_{yy}=\sum _{i=0}^n{(y_i-\overline{y})}^2\\ & =& S_T\end{array}$$ よって，全平方和$S_T$である$S_{yy}$は回帰直線から推定される値$\hat{y}$と目的変数の平均値$\overline{y}$との差の 平方和と残差平方和で表せられる．
この回帰直線から推定される値$\hat{y}$と目的変数の平均値$\overline{y}$との差の 平方和を，回帰による変動の平方和$S_R$と呼ぶ． $$\begin{array}{rcl}\sum _{i=0}^n{(y_i-\overline{y})}^2& =& \sum _{i=0}^n{(\widehat{y_i}-\overline{y})}^2+\sum _{i=0}^n{(y_i-\widehat{y_i})}^2\\ S_T& =& S_R+S_e\\ S_{yy}& =& S_R+S_e\\ S_{yy}& =& S_R+(S_{yy}-\frac{S_{xy}^2}{S_{xx}})\\ S_R& =& \frac{S_{xy}^2}{S_{xx}}\end{array}$$ 回帰による変動の平方和$S_R$もデータの各平方和($S_{xx},S_{yy}$)より求めることができる．