単回帰モデルの最小二乗推定量の分布
単回帰モデルの最小二乗推定量\(\hat{\alpha},\hat{\beta}\)の分布
単回帰モデル
$$ \begin{eqnarray} y_i&=&\alpha+\beta x_i +\epsilon_i\;(i=1,\cdots,n)\;\dots\;\epsilon_i \overset{iid}{\sim} \mathrm{N}\left(0,\sigma^2\right) \\\mathrm{E}\left[y_i\right]&=&\mathrm{E}\left[\alpha+\beta x_i +\epsilon_i\right] \\&=&\alpha+\beta x_i +\mathrm{E}\left[\epsilon_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-expected.html}{\mathrm{E}\left[X+t\right]=\mathrm{E}\left[X\right]+t} \\&=&\alpha+\beta x_i+0 \;\cdots\;\epsilon_i \overset{iid}{\sim} \mathrm{N}\left(0,\sigma^2\right) \\&=&\alpha+\beta x_i \\\mathrm{V}\left[y_i\right]&=&\mathrm{V}\left[\alpha+\beta x_i +\epsilon_i\right] \\&=&\mathrm{V}\left[\epsilon_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-variance.html}{\mathrm{V}\left[X+t\right]=\mathrm{V}\left[X\right]} \\&=&\sigma^2 \;\cdots\;\epsilon_i \overset{iid}{\sim} \mathrm{N}\left(0,\sigma^2\right) \\y_i&\sim&\mathrm{N}(\alpha+\beta x_i,\sigma^2) \end{eqnarray} $$ \(y_i\)は\(\mathrm{N}\left(\alpha+\beta x_i,\sigma^2\right)\)に従う確率変数である．\(\hat{\beta}\)を\(\sum_{i=1}^n c_iy_i\)の形で表す
推定量が\(\sum_{i=1}^n c_ix_i\;(x_i:標本,\;c_i:定数)\)の形で表現できるとき，この推定量を線形推定量(linear estimate)という．(よく知られる線形推定量の例として平均\(\bar{x}\)があり，\(\bar{x}=\sum_{i=1}^n \frac{1}{n} x_i\)で表現される) $$ \begin{eqnarray} \hat{\beta}&=&\frac{S_{xy}}{S_{xx}} \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{\hat{\beta}=\frac{S_{xy}}{S_{xx}}} ,\;S_{xx}=\sum_{i=1}^n\left(x_i-\bar{x}\right)^2,\;\bar{x}=\frac{1}{n}\sum_{i=1}^nx_i \\&=&\frac{1}{S_{xx}} \sum_{i=1}^n \left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) \\&=&\frac{1}{S_{xx}} \sum_{i=1}^n \left(x_i-\bar{x}\right)y_i \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/10/sxy.html}{\sum_{i=1}^n\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)=S_{xy}= \sum_{i=1}^n \left(x_i-\bar{x}\right)y_i} \\&=& \sum_{i=1}^n \frac{x_i-\bar{x}}{S_{xx}}y_i \\&=& \sum_{i=1}^n c_iy_i \;\cdots\;c_i=\frac{x_i-\bar{x}}{S_{xx}} \end{eqnarray} $$
\(\hat{\beta}\)の期待値を\(\sum_{i=1}^n c_iy_i\)から求めてみる
$$ \begin{eqnarray} \mathrm{E}\left[\sum_{i=1}^n c_iy_i\right] &=&\mathrm{E}\left[\sum_{i=1}^n \frac{x_i-\bar{x}}{S_{xx}}y_i\right] \\&=&\mathrm{E}\left[\frac{S_{xy}}{S_{xx}}\right] \;\cdots\;上記,\;\frac{S_{xy}}{S_{xx}}=\sum_{i=1}^n \frac{x_i-\bar{x}}{S_{xx}}y_i \\&=&\mathrm{E}\left[\hat{\beta}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{\hat{\beta}=\frac{S_{xy}}{S_{xx}}} \\&=&\beta \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\beta}\right]=\beta} \end{eqnarray} $$\(\hat{\beta}\)の分散を\(\sum_{i=1}^n c_iy_i\)から求めてみる
$$ \begin{eqnarray} \mathrm{V}\left[\sum_{i=1}^n c_iy_i\right] &=&\mathrm{V}\left[\sum_{i=1}^n \frac{x_i-\bar{x}}{S_{xx}}y_i\right] \\&=&\sum_{i=1}^n \mathrm{V}\left[\frac{x_i-\bar{x}}{S_{xx}}y_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-variance.html}{y_iは互いに独立\mathrm{Cov}\left[y_i, y_j\right]=0,\;互いに独立の場合\mathrm{V}\left[X+Y\right]=\mathrm{V}\left[X\right]+\mathrm{V}\left[Y\right]} \\&=&\sum_{i=1}^n \left(\frac{x_i-\bar{x}}{S_{xx}}\right)^2\mathrm{V}\left[y_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]} \\&=&\sum_{i=1}^n \left(\frac{x_i-\bar{x}}{S_{xx}}\right)^2\sigma^2 \\&=&\frac{\sigma^2}{S_{xx}^2}\sum_{i=1}^n \left(x_i-\bar{x}\right)^2 \\&=&\frac{\sigma^2}{S_{xx}^2}S_{xx} \;\cdots\;S_{xx}=\sum_{i=1}^n \left(x_i-\bar{x}\right)^2 \\&=&\frac{\sigma^2}{S_{xx}} \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\frac{S_{xy}}{S_{xx}}\right]=\frac{\sigma^2}{S_{xx}}}と同じ結果 \end{eqnarray} $$\(\hat{\beta}\)の分布
以上のように，\(\hat{\beta}\)は線形推定量であり，正規分布に従う\(y_i\)の定数倍の和で表すことができた．よって\(\hat{\beta}\)は同様に正規分布に従い，その期待値と分散はそれぞれ上記で求めたとおりである \(\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/zc1xc2y-2.html}{Z=c_1X+c_2Y(X\sim\mathrm{N}(\mu_1,\sigma_1^2),Y\sim\mathrm{N}(\mu_2,\sigma_2^2),Z\sim\mathrm{N}(c_1\mu_1+c_2\mu_2,c_1^2\sigma_1^2+c_2^2\sigma_2^2))}\)． $$ \begin{eqnarray} \hat{\beta}&\sim& \mathrm{N}\left(\beta,\frac{\sigma^2}{S_{xx}}\right) \end{eqnarray} $$\(\hat{\alpha}\)を\(\sum_{i=1}^n c_iy_i\)の形で表す
$$ \begin{eqnarray} \hat{\alpha}&=&\bar{y}-\hat{\beta}\bar{x} \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{\hat{\alpha}=\bar{y}-\hat{\beta}\bar{x}} \\&=&\sum_{i=1}^n\frac{1}{n}y_i-\frac{S_{xy}}{S_{xx}}\bar{x} \\&=&\sum_{i=1}^n\frac{1}{n}y_i-\left(\sum_{i=1}^n\frac{x_i-\bar{x}}{S_{xx}}y_i\right)\bar{x} \;\cdots\;上記,\;\frac{S_{xy}}{S_{xx}}=\sum_{i=1}^n \frac{x_i-\bar{x}}{S_{xx}}y_i \\&=&\sum_{i=1}^n\frac{1}{n}y_i-\bar{x}\sum_{i=1}^n\frac{x_i-\bar{x}}{S_{xx}}y_i \\&=&\sum_{i=1}^n\frac{1}{n}y_i-\sum_{i=1}^n\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}y_i \\&=&\sum_{i=1}^n\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)y_i \\&=& \sum_{i=1}^n c_iy_i \;\cdots\;c_i=\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}} \end{eqnarray} $$\(\hat{\alpha}\)の期待値を\(\sum_{i=1}^n c_iy_i\)から求めてみる
$$ \begin{eqnarray} \mathrm{E}\left[\sum_{i=1}^n c_iy_i\right] &=&\mathrm{E}\left[\sum_{i=1}^n\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)y_i\right] \\&=&\mathrm{E}\left[\sum_{i=1}^n\left(\frac{1}{n}y_i-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}y_i\right)\right] \\&=&\mathrm{E}\left[\sum_{i=1}^n\frac{1}{n}y_i-\sum_{i=1}^n\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}y_i\right] \\&=&\mathrm{E}\left[\sum_{i=1}^n\frac{1}{n}y_i-\sum_{i=1}^n\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}y_i\right] \\&=&\mathrm{E}\left[\sum_{i=1}^n\frac{1}{n}y_i\right]-\mathrm{E}\left[\sum_{i=1}^n\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}y_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-expected.html}{\mathrm{E}\left[X+Y\right]=\mathrm{E}\left[X\right]+\mathrm{E}\left[Y\right]} \\&=&\mathrm{E}\left[\frac{1}{n}\sum_{i=1}^ny_i\right]-\mathrm{E}\left[\bar{x}\sum_{i=1}^n\frac{\left(x_i-\bar{x}\right)}{S_{xx}}y_i\right] \\&=&\frac{1}{n}\mathrm{E}\left[\sum_{i=1}^ny_i\right]-\bar{x}\mathrm{E}\left[\sum_{i=1}^n\frac{\left(x_i-\bar{x}\right)}{S_{xx}}y_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-expected.html}{\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]} \\&=&\frac{1}{n}\sum_{i=1}^n\mathrm{E}\left[y_i\right]-\bar{x}\mathrm{E}\left[\frac{S_{xy}}{S_{xx}}\right] \;\cdots\;上記,\;\frac{S_{xy}}{S_{xx}}=\sum_{i=1}^n \frac{x_i-\bar{x}}{S_{xx}}y_i \\&=&\frac{1}{n}\sum_{i=1}^n\left(\alpha+\beta x_i\right)-\bar{x}\mathrm{E}\left[\frac{S_{xy}}{S_{xx}}\right] \;\cdots\;\mathrm{E}\left[y_i\right]=\alpha+\beta x_i \\&=&\frac{1}{n}\left(\alpha\sum_{i=1}^n1+\beta\sum_{i=1}^n x_i\right)-\bar{x}\mathrm{E}\left[\hat{\beta}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{\hat{\beta}=\frac{S_{xy}}{S_{xx}}} \\&=&\frac{1}{n}\left(n\alpha+\beta\;n\bar{x}\right)-\bar{x}\beta \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\beta}\right]=\beta} \\&=&\frac{1}{n}n\left(\alpha+\beta\bar{x}\right)-\bar{x}\beta \\&=&\left(\alpha+\beta\bar{x}\right)-\bar{x}\beta \\&=&\alpha \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\alpha}\right]=\alpha} \end{eqnarray} $$\(\hat{\alpha}\)の分散を\(\sum_{i=1}^n c_iy_i\)から求めてみる
$$ \begin{eqnarray} \mathrm{V}\left[\sum_{i=1}^n c_iy_i\right] &=&\mathrm{V}\left[\sum_{i=1}^n\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)y_i\right] \\&=&\sum_{i=1}^n\mathrm{V}\left[\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)y_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\frac{S_{xy}}{S_{xx}}\right]=\frac{\sigma^2}{S_{xx}}}と同じ結果 \\&=&\sum_{i=1}^n\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)^2\mathrm{V}\left[y_i\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/continuous-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]} \\&=&\sum_{i=1}^n\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)^2\sigma^2 \\&=&\sigma^2\sum_{i=1}^n\left(\frac{1}{n}-\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)^2 \\&=&\sigma^2\sum_{i=1}^n\left( \frac{1}{n^2} -2\frac{1}{n}\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}} +\left(\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)^2 \right) \\&=&\sigma^2\left( \sum_{i=1}^n\frac{1}{n^2} -\sum_{i=1}^n2\frac{1}{n}\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}} +\sum_{i=1}^n\left(\frac{\bar{x}\left(x_i-\bar{x}\right)}{S_{xx}}\right)^2 \right) \\&=&\sigma^2\left( \frac{1}{n^2}\sum_{i=1}^n1 -\frac{2\bar{x}}{nS_{xx}}\sum_{i=1}^n\left(x_i-\bar{x}\right) +\frac{\bar{x}^2}{S_{xx}^2}\sum_{i=1}^n\left(x_i-\bar{x}\right)^2 \right) \\&=&\sigma^2\left( \frac{1}{n^2}n -\frac{2\bar{x}}{nS_{xx}}\cdot 0 +\frac{\bar{x}^2}{S_{xx}^2}S_{xx} \right) \\&=&\sigma^2\left( \frac{1}{n^2}\cdot n -\frac{2\bar{x}}{nS_{xx}}\cdot 0 +\frac{\bar{x}^2}{S_{xx}^2}\cdot S_{xx} \right) \\&=&\sigma^2\left( \frac{1}{n} +\frac{\bar{x}^2}{S_{xx}} \right) \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\bar{y}-\hat{\beta}\bar{x}\right]=\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2と同じ結果} \end{eqnarray} $$\(\hat{\alpha}\)の分布
以上のように，\(\hat{\alpha}\)は線形推定量であり，正規分布に従う\(y_i\)の定数倍の和で表すことができた．よって\(\hat{\alpha}\)は同様に正規分布に従い，その期待値と分散はそれぞれ上記で求めたとおりである \(\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/zc1xc2y-2.html}{Z=c_1X+c_2Y(X\sim\mathrm{N}(\mu_1,\sigma_1^2),Y\sim\mathrm{N}(\mu_2,\sigma_2^2),Z\sim\mathrm{N}(c_1\mu_1+c_2\mu_2,c_1^2\sigma_1^2+c_2^2\sigma_2^2))}\)． $$ \begin{eqnarray} \hat{\alpha}&\sim& \mathrm{N}\left(\alpha,\sigma^2\left( \frac{1}{n} +\frac{\bar{x}^2}{S_{xx}} \right)\right) \end{eqnarray} $$単回帰モデルの最尤推定量の期待値,分散,分布
単回帰モデルの最尤推定量の期待値,分散,分布
単回帰モデル
$$ \begin{eqnarray} y_i&=&\alpha+\beta x_i+\epsilon_i \;(i=1,\cdots,n) \\&&\epsilon_i \overset{iid}{\sim} N(0,\sigma^2)\;\cdots\;独立同一分布(independent\;and\;identically\;distributed;\;IID,\;i.i.d.,\;iid) \end{eqnarray} $$ \(\alpha,\beta,\sigma^2\)の推定量を\(\hat{\alpha},\hat{\beta},\hat{\sigma}^2\)とし，\(\hat{\alpha},\hat{\beta},\hat{\sigma}^2\)の最尤推定量(maximum likelihood estimator)を\(\hat{\alpha}_{ML},\hat{\beta}_{ML},\hat{\sigma}^2_{ML}\)とする．\(\hat{\beta}_{ML}\)の期待値
$$ \begin{eqnarray} \mathrm{E}\left[\hat{\beta}_{ML}\right]&=&\mathrm{E}\left[\hat{\beta}\right] \;\cdots\;\hat{\beta}_{ML}=\hat{\beta} \\&=&\beta \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\beta}\right]=\beta} \\&&\;\cdots\;よって\hat{\beta}_{ML}^2は\beta^2の不偏推定量で\mathbf{ある}. \end{eqnarray} $$\(\hat{\alpha}_{ML}\)の期待値
$$ \begin{eqnarray} \mathrm{E}\left[\hat{\alpha}_{ML}\right]&=&\mathrm{E}\left[\hat{\alpha}\right] \;\cdots\;\hat{\alpha}_{ML}=\hat{\alpha} \\&=&\alpha \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2.html}{\mathrm{E}\left[\hat{\alpha}\right]=\alpha} \\&&\;\cdots\;よって\hat{\alpha}_{ML}^2は\alpha^2の不偏推定量で\mathbf{ある}. \end{eqnarray} $$\(\hat{\beta}_{ML}\)の分散
$$ \begin{eqnarray} \mathrm{V}\left[\hat{\beta}_{ML}\right]&=&\mathrm{V}\left[\hat{\beta}\right] \\&=&\frac{1}{S_{xx}}\sigma^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\hat{\beta}\right]=\frac{1}{S_{xx}}\sigma^2} \\&&\;\cdots\;\bar{x}=\frac{1}{n}\sum_{i=0}^{n}x_i,\;S_{xx}=\sum_{i=0}^{n}\left(x_i-\bar{x}\right)^2 \end{eqnarray} $$\(\hat{\alpha}_{ML}\)の分散
$$ \begin{eqnarray} \mathrm{V}\left[\hat{\alpha}_{ML}\right]&=&\mathrm{V}\left[\hat{\alpha}\right] \\&=&\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/2variancecovariance.html}{\mathrm{V}\left[\hat{\alpha}\right]=\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2} \end{eqnarray} $$\(\hat{\alpha}_{ML},\hat{\beta}_{ML}\)の分布
$$ \begin{eqnarray} \hat{\beta}_{ML}&=&\hat{\beta}&\sim&\mathrm{N}\left(\beta,\;\frac{1}{S_{xx}}\sigma^2\right) \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post_29.html}{\hat{\beta}\sim\mathrm{N}\left(\beta,\;\frac{1}{S_{xx}}\sigma^2\right)} \\\hat{\alpha}_{ML}&=&\hat{\alpha}&\sim&\mathrm{N}\left(\alpha,\;\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2\right) \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post_29.html}{\hat{\alpha}\sim\mathrm{N}\left(\alpha,\;\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2\right)} \end{eqnarray} $$\(\hat{\sigma}^2_{ML}\)の期待値
$$ \begin{eqnarray} \mathrm{E}\left[\hat{\sigma}_{ML}^2\right]&=&\mathrm{E}\left[\frac{n-2}{n}s^2\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post_25.html}{\hat{\sigma}^2_{ML}=\frac{n-2}{n}s^2} ,\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{s^2=\frac{1}{\left(n-2\right)}\sum_{i=1}^{n} e_i^2} ,\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{\sum_{i=1}^{n}e_i^2=\sum_{i=1}^{n}\left(y_i-\hat{y_i}\right)^2} \\&=&\frac{n-2}{n}\mathrm{E}\left[s^2\right] \\&=&\frac{n-2}{n}\sigma^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{\mathrm{E}\left[s^2\right]=\sigma^2} \\&\lt&\sigma^2 \;\cdots\;よって\hat{\sigma}_{ML}^2は\sigma^2の不偏推定量では\mathbf{ない}. \end{eqnarray} $$確率変数の標準化
確率変数の標準化
期待値(平均)が\(\mu\), 分散が\(\sigma^2\)の確率変数\(X\)
$$ \begin{eqnarray} \mathrm{E}\left[X\right]&=&\mu \\\mathrm{V}\left[X\right]&=&\sigma^2 \end{eqnarray} $$確率変数の変換\(Z=\frac{X-\mu}{\sigma}\)
$$ \begin{eqnarray} \\Z&=&\frac{X-\mu}{\sigma} \end{eqnarray} $$変換後の確率変数\(Z\)の期待値(平均)と分散
$$ \begin{eqnarray} \\\mathrm{E}\left[Z\right]&=&\mathrm{E}\left[\frac{X-\mu}{\sigma}\right] \\&=&\frac{1}{\sigma}\mathrm{E}\left[X-\mu\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]} \\&=&\frac{1}{\sigma}\left(\mathrm{E}\left[X\right]-\mu\right) \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[X\pm t\right]=\mathrm{E}\left[X\right] \pm t} \\&=&\frac{1}{\sigma}\left(\mu-\mu\right) \;\cdots\;\mathrm{E}\left[X\right]=\mu \\&=&\frac{1}{\sigma}\left(0\right) \\&=&0 \\\mathrm{V}\left[Z\right]&=&\mathrm{V}\left[\frac{X-\mu}{\sigma}\right] \\&=&\frac{1}{\sigma^2}\mathrm{V}\left[X-\mu\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]} \\&=&\frac{1}{\sigma^2}\mathrm{V}\left[X\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[X\pm t\right]=\mathrm{V}\left[X\right]} \\&=&\frac{1}{\sigma^2}\sigma^2 \;\cdots\;\mathrm{V}\left[X\right]=\sigma^2 \\&=&1 \end{eqnarray} $$ \(X\)の分布によらず，\(X\)の期待値(平均)と分散が\(\mu\)と\(\sigma^2\)であることから\(Z\)の期待値(平均)と分散が\(0\), \(1\)と標準化される．確率母凾数 ( probability generating function )
確率母凾数
非負の整数値をとる離散型確率変数\(X\)に対して以下のように確率母凾数(probability generating function;積率母凾数ではない)が定義される． $$ \begin{eqnarray} G_X(t) &=&\mathrm{E}\left[t^X\right] \;\cdots\;積率母凾数はM_X(t)=\mathrm{E}\left[e^{tX}\right] \\&=&\sum_{k} t^k P(X=k) \\&&\;\cdots\;P(X):確率質量凾数, \sum_{k}:Xの定義範囲すべてのkでの和 \end{eqnarray} $$一階微分((原点周りの)一次モーメント) = 期待値
$$ \begin{eqnarray} \left. G_X^{(1)}(t) \right|_{t=1} &=&\left. \frac{\mathrm{d}}{\mathrm{d}t}G_X(t) \right|_{t=1} \\&=&\left. \sum_{k\geq1} kt^{k-1} P(X=k) \right|_{t=1} \\&=&\sum_{k\geq1} k1^{k-1} P(X=k) \\&=&\sum_{k\geq1} k P(X=k) \\&=&\mathrm{E}\left[X\right] \end{eqnarray} $$二階微分((原点周りの)二次モーメント)
$$ \begin{eqnarray} \left. G_X^{(2)}(t) \right|_{t=1} &=&\left. \frac{\mathrm{d}^2}{\mathrm{d}t^2}G_X(t) \right|_{t=1} \\&=&\left. \frac{\mathrm{d}}{\mathrm{d}t} \sum_{k\geq1} kt^{k-1} P(X=k) \right|_{t=1} \\&=&\left. \sum_{k\geq2} k(k-1)t^{k-2} P(X=k) \right|_{t=1} \\&=&\sum_{k\geq2} k(k-1)1^{k-2} P(X=k) \\&=&\sum_{k\geq2} k(k-1) P(X=k) \\&=&\mathrm{E}\left[X(X-1)\right] \end{eqnarray} $$分散((母平均周りの)二次モーメント)
$$ \begin{eqnarray} \mathrm{V}\left[X\right] &=&\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)^2\right] \\&=&\mathrm{E}\left[X^2\right]-\mathrm{E}\left[X\right]^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)^2\right]=\mathrm{E}\left[X^2\right]-\left[X\right]^2} \\&=&\mathrm{E}\left[X^2\right]\color{red}{-\mathrm{E}\left[X\right]+\mathrm{E}\left[X\right]}\color{black}{-\mathrm{E}\left[X\right]^2} \\&=&\mathrm{E}\left[X^2-X\right]+\mathrm{E}\left[X\right]-\mathrm{E}\left[X\right]^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[X\pm Y\right]=\mathrm{E}\left[X\right]\pm\mathrm{E}\left[Y\right]} \\&=&\mathrm{E}\left[X(X-1)\right]+\mathrm{E}\left[X\right]-\mathrm{E}\left[X\right]^2 \\&=&\left( \left. G_X^{(2)}(t) \right|_{t=1} \right) + \left( \left. G_X^{(1)}(t) \right|_{t=1} \right) - \left( \left. G_X^{(1)}(t) \right|_{t=1} \right)^2 \end{eqnarray} $$Z=X+Y
$$ \begin{eqnarray} G_Z(t)&=&\mathrm{E}\left[t^Z\right] \\&=&\mathrm{E}\left[t^{X+Y}\right] \\&=&\mathrm{E}\left[t^Xt^Y\right] \;\cdots\;A^{B+C}=A^BA^C \\&=&\mathrm{E}\left[t^X\right]\mathrm{E}\left[t^Y\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[AB\right]=\mathrm{E}\left[A\right]\mathrm{E}\left[B\right]\;AとBが独立の場合} \\&=&G_X(t)G_Y(t) \end{eqnarray} $$二項分布(binomial distribution)の積率母凾数(moment-generating function)と期待値(expected value)・分散(variance)
二項分布(binomial distribution)の積率母凾数(moment-generating function)と期待値(expected value)・分散(variance)
二項分布
$$ \begin{eqnarray} X&\sim&B(n, p) \\f(X=x)&=& \begin{cases} _n\mathrm{C}_x\;p^x(1-p)^{n-x} & x \in \left\{0,1,2, \dotsc ,n\right\} \\0 & x \notin \left\{0,1,2, \dotsc ,n\right\} \end{cases}\;\cdots\;確率密度凾数 \end{eqnarray} $$積率母凾数
$$ \begin{eqnarray} M_X(t)&=&\mathrm{E}\left[e^{tx}\right] \\&=&\sum_{k=0}^n e^{tx}\;_n\mathrm{C}_x\;p^x\left(1-p\right)^{n-x} \\&=&\sum_{k=0}^n\;_n\mathrm{C}_x\;\left( e^{t} p \right)^x \left(1-p\right)^{n-x} \\&=&\;_n\mathrm{C}_0\;\left( e^{t} p \right)^0 \left(1-p\right)^{n-0} +\;_n\mathrm{C}_1\;\left( e^{t} p \right)^1 \left(1-p\right)^{n-1} +\;_n\mathrm{C}_2\;\left( e^{t} p \right)^2 \left(1-p\right)^{n-2} +\cdots +\;_n\mathrm{C}_n\;\left( e^{t} p \right)^n \left(1-p\right)^{n-n} \\&=&\left\{e^{t} p + \left(1-p\right) \right\}^n \\&&\;\cdots\;(A+B)^D=\;_D\mathrm{C}_0\;A^0B^{D-0}+\;_D\mathrm{C}_1\;A^1B^{D-1}+\cdots+\;_D\mathrm{C}_D\;A^DB^{D-D}=\sum_{k=0}^D \;_D\mathrm{C}_k\;A^kB^{D-k}\;(二項関係) \\&=&\left(e^{t}p - p + 1\right)^n \end{eqnarray} $$(原点周りの)一次モーメント = 期待値
$$ \begin{eqnarray} \mathrm{E}\left[X\right]&=&M^{(1)}_X(t) \\&=&\left.\frac{\mathrm{d} M_X(t)}{\mathrm{d}t}\right|_{t=0} \\&=&\left.\frac{\mathrm{d}}{\mathrm{d}t} \left(e^{t}p - p + 1\right)^n \right|_{t=0} \\&=&\left.\frac{\mathrm{d}}{\mathrm{d}u} u^n \frac{\mathrm{d}u}{\mathrm{d}t} \right|_{t=0} \;\cdots\;u=e^{t}p - p + 1,\frac{\mathrm{d}u}{\mathrm{d}t}=\frac{\mathrm{d}}{\mathrm{d}t}\left(e^{t}p - p + 1\right)=e^{t}p \\&=&\left.nu^{n-1} e^{t}p \right|_{t=0} \\&=&\left.n\left( e^{t}p - p + 1 \right)^{n-1} e^{t}p \right|_{t=0} \\&=&\left.npe^{t} \left( e^{t}p - p + 1 \right)^{n-1} \right|_{t=0} \\&=&npe^{0} \left( e^{0}p - p + 1 \right)^{n-1} \\&=&np\cdot1 \left( 1\cdot p - p + 1 \right)^{n-1} \\&=&np \left(1\right)^{n-1} \\&=&np\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/blog-post_30.html}{Xの期待値の定義から求めた二項分布の期待値}と同じ \end{eqnarray} $$(原点周りの)二次モーメント
$$ \begin{eqnarray} \mathrm{E}\left[X^2\right]&=&M^{(2)}_X(t) \\&=&\left.\frac{\mathrm{d}^2 M_X(t)}{\mathrm{d}t^2}\right|_{t=0} \\&=&\left.\frac{\mathrm{d}^2}{\mathrm{d}t^2} \left(e^{t}p - p + 1\right)^n \right|_{t=0} \\&=&\left.\frac{\mathrm{d}}{\mathrm{d}t} npe^{t} \left(e^{t}p - p + 1\right)^{n-1} \right|_{t=0} \\&=&\left.np\frac{\mathrm{d}}{\mathrm{d}t} e^{t} \left(e^{t}p - p + 1\right)^{n-1} \right|_{t=0} \;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x)\;(c:定数) \\&=&\left.np\frac{\mathrm{d}}{\mathrm{d}t} uv \right|_{t=0} \;\cdots\;u=e^t,\;v= \left(e^{t}p - p + 1\right)^{n-1} \\&=&\left.np\left\{\left(\frac{\mathrm{d}}{\mathrm{d}t}u\right)v+u\left(\frac{\mathrm{d}}{\mathrm{d}t}v\right) \right\}\right|_{t=0} \\&=&\left.np\left[\left(e^{t}\right)v+u\left\{\left(n-1\right)\left(e^{t}p - p + 1\right)^{n-2}pe^t\right\} \right]\right|_{t=0} \;\cdots\;\frac{\mathrm{d}u}{\mathrm{d}t}=e^{t},\frac{\mathrm{d}v}{\mathrm{d}t}=\left(n-1\right)\left(e^{t}p - p + 1\right)^{n-2}pe^t \\&=&\left.np\left[\left(e^{t}\right)\left(e^{t}p - p + 1\right)^{n-1}+e^t\left\{\left(n-1\right)\left(e^{t}p - p + 1\right)^{n-2}pe^t\right\} \right]\right|_{t=0} \;\cdots\;u=e^t,\;v= \left(e^{t}p - p + 1\right)^{n-1} \\&=&\left.npe^{t}\left(e^{t}p - p + 1\right)^{n-1}+n\left(n-1\right)p^2e^{2t}\left(e^{t}p - p + 1\right)^{n-2} \right|_{t=0} \\&=&npe^{0}\left(e^{0}p - p + 1\right)^{n-1}+n\left(n-1\right)p^2e^{2\cdot0}\left(e^{0}p - p + 1\right)^{n-2} \\&=&np\cdot1\cdot\left(1\cdot p - p + 1\right)^{n-1}+n\left(n-1\right)p^2\cdot1\cdot\left(1\cdot p - p + 1\right)^{n-2} \\&=&np\left(1\right)^{n-1}+n\left(n-1\right)p^2\left(1\right)^{n-2} \\&=&np+n\left(n-1\right)p^2 \end{eqnarray} $$二次の中心(化)モーメント / 母平均周りの二次モーメント = 分散
$$ \begin{eqnarray} \mathrm{V}\left[X\right]&=&\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)^2\right] \\&=&\mathrm{E}\left[X^2\right]-\mathrm{E}\left[X\right]^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)^2\right]=\mathrm{E}\left[X^2\right]-\left[X\right]^2} \\&=&M^{(2)}_X(t) -\left(M^{(1)}_X(t)\right)^2 \\&=&np+n\left(n-1\right)p^2 - (np)^2 \\&=&np+n^2p^2-np^2 - n^2p^2 \\&=&np-np^2 \\&=&np(1-p)\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/blog-post_75.html}{X(X-1)の期待値を利用した二項分布の分散}と同じ \end{eqnarray} $$単回帰における最小二乗推定量の分散(variance)・共分散(covariance)
単回帰における最小二乗推定量\(\hat{\alpha},\;\hat{\beta}\)の分散(variance)・共分散(covariance)
単回帰における観測値\(y_i\)の分散・共分散について
$$ \begin{eqnarray} y_i&=&\alpha+\beta x_i+\epsilon_i\;(i=1,\cdots,n) \\\left\{\epsilon_i|i=1,\cdots,n\right\}&:&\epsilon_i \overset{iid}{\sim} N(0,\sigma^2) \\&&\;\cdots\;独立同一分布(independent\;and\;identically\;distributed;\;IID,\;i.i.d.,\;iid) \\&&\;\cdots\;\mathrm{E}\left[\epsilon_i\right]=0,\;\mathrm{V}\left[\epsilon_i\right]=\sigma^2,互いに独立\left(\mathrm{Cov}[\epsilon_i, \epsilon_j]=\left\{\begin{array}\;\mathrm{V}\left[\epsilon_i\right]=\sigma^2&(i=j)\\0&(i \neq j)\end{array}\right.\right) \\\mathrm{V}\left[y_i\right] &=&\mathrm{V}\left[\alpha+\beta x_i+\epsilon_i\right] \\&=&\mathrm{V}\left[\epsilon_i\right] \;\cdots\;\mathrm{V}\left[X\pm t\right]=\mathrm{V}\left[X\right]\;(t:分散をとることについて定数) \\&=&\sigma^2 \\\mathrm{Cov}\left[y_i, y_j\right] &=&\mathrm{E}\left[\left(y_i-\mathrm{E}\left[y_i\right]\right)\left(y_j-\mathrm{E}\left[y_j\right]\right)\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{Cov}\left[X,Y\right]=\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)\left(Y-\mathrm{E}\left[Y\right]\right)\right]} \\&=&\mathrm{E}\left[\left(\alpha+\beta x_i+\epsilon_i-\mathrm{E}\left[\alpha+\beta x_i+\epsilon_i\right]\right)\left(\alpha+\beta x_j+\epsilon_j-\mathrm{E}\left[\alpha+\beta x_j+\epsilon_j\right]\right)\right] \;\cdots\;y_i=\alpha+\beta x_i+\epsilon_i \\&=&\mathrm{E}\left[\left(\alpha+\beta x_i+\epsilon_i-\alpha-\beta x_i-\mathrm{E}\left[\epsilon_i\right]\right)\left(\alpha+\beta x_j+\epsilon_j-\alpha-\beta x_j-\mathrm{E}\left[\epsilon_j\right]\right)\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[X\pm t\right]=\mathrm{E}\left[X\right]\pm t} \\&=&\mathrm{E}\left[\left(\epsilon_i-\mathrm{E}\left[\epsilon_i\right]\right)\left(\epsilon_j-\mathrm{E}\left[\epsilon_j\right]\right)\right] \\&=&\mathrm{Cov}\left[\epsilon_i, \epsilon_j\right] \end{eqnarray} $$ 上記を踏まえて\((x_i-\bar{x})\)を加えた分散・共分散について $$ \begin{eqnarray} \mathrm{Cov}\left[(x_i-\bar{x})(y_i-\bar{y}), (x_j-\bar{x})(y_j-\bar{y})\right] &=&(x_i-\bar{x})(x_j-\bar{x})\mathrm{Cov}\left[(y_i-\bar{y}), (y_j-\bar{y})\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{Cov}\left[c_0X_i, c_1X_j\right]=c_0c_1\mathrm{Cov}\left[X_i,X_j\right]} \\&=&(x_i-\bar{x})(x_j-\bar{x})\mathrm{E}\left[\left\{(y_i-\bar{y})-\mathrm{E}\left[y_i-\bar{y}\right]\right\}\left\{(y_j-\bar{y})-\mathrm{E}\left[y_j-\bar{y}\right]\right\}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{Cov}\left[X,Y\right]=\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)\left(Y-\mathrm{E}\left[Y\right]\right)\right]} \\&=&(x_i-\bar{x})(x_j-\bar{x})\mathrm{E}\left[\left\{y_i-\bar{y}-\mathrm{E}\left[y_i\right]+\bar{y}\right\}\left\{y_j-\bar{y}-\mathrm{E}\left[y_j\right]+\bar{y}\right\}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[X\pm t\right]=\mathrm{E}\left[X\right]\pm t} \\&=&(x_i-\bar{x})(x_j-\bar{x})\mathrm{E}\left[\left(y_i-\mathrm{E}\left[\bar{y}\right]\right)\left(y_j-\mathrm{E}\left[\bar{y}\right]\right)\right] \\&=&(x_i-\bar{x})(x_j-\bar{x})\mathrm{Cov}\left[y_i, y_j\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{Cov}\left[X,Y\right]=\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)\left(Y-\mathrm{E}\left[Y\right]\right)\right]} \\&=&\left\{\begin{array} \;(x_i-\bar{x})^2\sigma^2&(i=j) \\(x_i-\bar{x})(x_j-\bar{x})0&(i \neq j) \end{array}\right. \;\cdots\;\mathrm{Cov}\left[y_i,y_j\right]=\mathrm{Cov}\left[\epsilon_i,\epsilon_j\right]=\left\{\begin{array}\;\mathrm{V}\left[\epsilon_i\right]=\sigma^2&(i=j)\\0&(i \neq j)\end{array}\right. \\ \mathrm{V}\left[(x_i-\bar{x})y_i\right] &=&(x_i-\bar{x})^2\mathrm{V}\left[y_i\right]\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]} \\&=&(x_i-\bar{x})^2\sigma^2\;\cdots\;上記i=jのケース \end{eqnarray} $$\(S_{xy}\)の分散
$$ \begin{eqnarray} \mathrm{V}\left[S_{xy}\right] &=&\mathrm{V}\left[\sum_{i=1}^{n}\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)\right] \;\cdots\;S_{xy}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) \\&=& \sum_{i=1}^{n}\mathrm{V}\left[\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)\right] +2\sum_{i\lt j}\mathrm{Cov}\left[\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right), \left(x_j-\bar{x}\right)\left(y_j-\bar{y}\right)\right] \\&&\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{V}\left[\sum_{i=1}^{n}X_i\right] =\sum_{i=1}^{n}\mathrm{V}\left[X_i\right]+2\sum_{i\lt j}\mathrm{Cov}\left[X_i, X_j\right]} \\&=&\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2\sigma^2+2\sum_{i\lt j}0 \\&&\;\cdots\;\mathrm{V}\left[\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)\right]=\left(x_i-\bar{x}\right)^2\sigma^2 \\&&\;\cdots\;\mathrm{Cov}\left[\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right), \left(x_j-\bar{x}\right)\left(y_j-\bar{y}\right)\right]=0\;(i\neq j) \\&=&\sigma^2\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2 \;\cdots\;\sum_{i=0}^n cX_i=c\sum_{i=0}^n X_i \\&=&\sigma^2S_{xx} \;\cdots\;S_{xx}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2 \end{eqnarray} $$最小2乗推定量\(\hat{\beta}\)の分散
$$ \begin{eqnarray} \mathrm{V}\left[\hat{\beta}\right] &=&\mathrm{V}\left[\frac{S_{xy}}{S_{xx}}\right] \;\cdots\;\hat{\beta}=\frac{S_{xy}}{S_{xx}},\;S_{xx}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2,\;S_{xy}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) \\&=&\frac{1}{S_{xx}^2}\mathrm{V}\left[S_{xy}\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]} \\&=&\frac{1}{S_{xx}^2}\sigma^2S_{xx} \;\cdots\;\mathrm{V}\left[S_{xy}\right]=\sigma^2S_{xx} \\&=&\frac{1}{S_{xx}}\sigma^2 \end{eqnarray} $$最小2乗推定量\(\hat{\alpha}\)の分散
$$ \begin{eqnarray} \mathrm{V}\left[\hat{\alpha}\right] &=&\mathrm{V}\left[\bar{y}-\hat{\beta}\bar{x}\right] \;\cdots\;\hat{\alpha}=\bar{y}-\hat{\beta}\bar{x} \\&=&\mathrm{V}\left[\bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x}\right] \;\cdots\;\hat{\beta}=\frac{S_{xy}}{S_{xx}},\;S_{xx}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2,\;S_{xy}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) \\&=&\mathrm{V}\left[\bar{y}\right]+\mathrm{V}\left[\frac{S_{xy}}{S_{xx}}\bar{x}\right] -2\mathrm{Cov}\left[\bar{y}, \frac{S_{xy}}{S_{xx}}\bar{x}\right] \\&&\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[X\pm Y\right]=\mathrm{V}\left[X\right]\pm 2\mathrm{Cov}\left[X,Y\right]+\mathrm{V}\left[Y\right]} \\&=&\mathrm{V}\left[\bar{y}\right]+\mathrm{V}\left[\frac{S_{xy}}{S_{xx}}\bar{x}\right]-2\cdot0 \;\cdots\;\mathrm{Cov}\left[\bar{y}, \frac{S_{xy}}{S_{xx}}\bar{x}\right]=0\;(後述) \\&=&\mathrm{V}\left[\bar{y}\right] +\frac{\bar{x}^2}{S_{xx}^2}\mathrm{V}\left[S_{xy}\right] \\&=&\mathrm{V}\left[\frac{1}{n}\sum_{i=1}^{n}y_i\right] +\frac{\bar{x}^2}{S_{xx}^2}\sigma^2S_{xx} \;\cdots\;\mathrm{V}\left[S_{xy}\right]=\sigma^2S_{xx} \\&=&\frac{1}{n^2}\mathrm{V}\left[\sum_{i=1}^{n}y_i\right] +\frac{\bar{x}^2}{S_{xx}}\sigma^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-variance.html}{\mathrm{V}\left[cX\right]=c^2\mathrm{V}\left[X\right]} \\&=&\frac{1}{n^2} \left\{ \sum_{i=1}^{n}\mathrm{V}\left[y_i\right] +2\sum_{i\lt j}\mathrm{Cov}\left[y_i, y_j\right] \right\} +\frac{\bar{x}^2}{S_{xx}}\sigma^2 \\&&\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{V}\left[\sum_{i=1}^{n}X_i\right] =\sum_{i=1}^{n}\mathrm{V}\left[X_i\right]+2\sum_{i\lt j}\mathrm{Cov}\left[X_i, X_j\right]} \\&=&\frac{1}{n^2}\left\{ \sum_{i=1}^{n}\sigma^2 +2\sum_{i\lt j}0 \right\} +\frac{\bar{x}^2}{S_{xx}}\sigma^2 \;\cdots\;\mathrm{V}\left[y_i\right]=\sigma^2,\;\mathrm{Cov}\left[y_i, y_j\right]=0 \\&=&\frac{1}{n^2}n\sigma^2 +\frac{\bar{x}^2}{S_{xx}}\sigma^2 \;\cdots\;\sum_{i=0}^n c=nc \\&=&\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}}\right)\sigma^2 \end{eqnarray} $$最小2乗推定量\(\hat{\alpha}\)と\(\hat{\beta}\)の共分散
$$ \begin{eqnarray} \mathrm{Cov}\left[\hat{\alpha},\hat{\beta}\right] &=&\mathrm{E}\left[\left(\hat{\alpha}-\mathrm{E}\left[\hat{\alpha}\right]\right)\left(\hat{\beta}-\mathrm{E}\left[\hat{\beta}\right]\right)\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{Cov}\left[X,Y\right]=\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)\left(Y-\mathrm{E}\left[Y\right]\right)\right]} \\&=&\mathrm{E}\left[ \left( \left(\bar{y}-\hat{\beta}\bar{x}\right) -\mathrm{E}\left[\bar{y}-\hat{\beta}\bar{x}\right] \right) \left( \hat{\beta}-\mathrm{E}\left[\hat{\beta}\right] \right) \right] \;\cdots\;\alpha=\bar{y}-\hat{\beta}\bar{x} \\&=&\mathrm{E}\left[ \left( \bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x} -\mathrm{E}\left[ \bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x} \right] \right) \left( \frac{S_{xy}}{S_{xx}} -\mathrm{E}\left[ \frac{S_{xy}}{S_{xx}} \right] \right) \right] \;\cdots\;\hat{\beta}=\frac{S_{xy}}{S_{xx}},\;S_{xx}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2,\;S_{xy}=\sum_{i=1}^{n}\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) \\&=&\mathrm{E}\left[ \left( \bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x} -\mathrm{E}\left[ \bar{y} \right] +\mathrm{E}\left[ \frac{S_{xy}}{S_{xx}}\bar{x} \right] \right) \left( \frac{S_{xy}}{S_{xx}} -\mathrm{E}\left[ \frac{S_{xy}}{S_{xx}} \right] \right) \right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[X\pm Y\right]=\mathrm{E}\left[X\right]\pm\mathrm{E}\left[Y\right]} \\&=&\mathrm{E}\left[ \left( \bar{y}-\frac{S_{xy}}{S_{xx}}\bar{x} -\mathrm{E}\left[ \bar{y} \right] +\frac{\bar{x}}{S_{xx}}\mathrm{E}\left[ S_{xy} \right] \right) \left( \frac{S_{xy}}{S_{xx}} -\frac{1}{S_{xx}}\mathrm{E}\left[ S_{xy} \right] \right) \right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]} \\&=&\mathrm{E}\left[ \left\{ \bar{y}-\mathrm{E}\left[\bar{y}\right] -\frac{\bar{x}}{S_{xx}}\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right) \right\} \left\{ \frac{1}{S_{xx}}\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right) \right\} \right] \\&=&\mathrm{E}\left[ -\frac{\bar{x}}{S_{xx}}\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right) \frac{1}{S_{xx}}\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right) \right] \;\cdots\;\bar{y}-\mathrm{E}\left[\bar{y}\right]=\bar{y}-\bar{y}=0 \\&=&\mathrm{E}\left[ -\frac{\bar{x}}{S_{xx}^2}\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right)^2 \right] \\&=&-\frac{\bar{x}}{S_{xx}^2}\mathrm{E}\left[ \left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right)^2 \right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]} \\&=&-\frac{\bar{x}}{S_{xx}^2}\sigma^2S_{xx} \;\cdots\;\mathrm{E}\left[\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right)^2\right]=\mathrm{V}\left[S_{xy}\right]=\sigma^2S_{xx} \\&=&-\frac{\bar{x}}{S_{xx}}\sigma^2 \end{eqnarray} $$\(\mathrm{Cov}\left[\bar{y}, \frac{S_{xy}}{S_{xx}}\bar{x}\right]=0\)について
$$ \begin{eqnarray} \mathrm{Cov}\left[\bar{y}, \frac{S_{xy}}{S_{xx}}\bar{x}\right] &=&\mathrm{E}\left[\left(\bar{y}-\mathrm{E}\left[\bar{y}\right]\right)\left(\frac{S_{xy}}{S_{xx}}\bar{x}-\mathrm{E}\left[\frac{S_{xy}}{S_{xx}}\bar{x}\right]\right)\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/covariance.html}{\mathrm{Cov}\left[X,Y\right]=\mathrm{E}\left[\left(X-\mathrm{E}\left[X\right]\right)\left(Y-\mathrm{E}\left[Y\right]\right)\right]} \\&=&\mathrm{E}\left[\left(\bar{y}-\bar{y}\right)\left(\frac{S_{xy}}{S_{xx}}\bar{x}-\frac{\bar{x}}{S_{xx}}\mathrm{E}\left[S_{xy}\right]\right)\right] \;\cdots\;\mathrm{E}\left[\bar{y}\right]=\bar{y},\;\href{https://shikitenkai.blogspot.com/2019/06/discrete-random-variable-expected-value.html}{\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]} \\&=&\mathrm{E}\left[0\cdot\frac{\bar{x}}{S_{xx}}\left(S_{xy}-\mathrm{E}\left[S_{xy}\right]\right)\right] \\&=&\mathrm{E}\left[0\right] \\&=&0 \end{eqnarray} $$カイ二乗分布の期待値と分散
カイ二乗分布の期待値と分散
カイ二乗分布の期待値(一次モーメント)
$$ \begin{eqnarray} \mathrm{E}\left[x\right]&=&\int_0^\infty x \chi^2(x) \mathrm{d}x \\&=&\int_0^\infty x \frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}e^{-\frac{x}{2}}x^{\frac{n}{2}-1} \mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}\int_0^\infty x e^{-\frac{x}{2}}x^{\frac{n}{2}-1} \mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}} \mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}} \color{red}{2^{\frac{n}{2}}\left(\frac{1}{2}\right)^{\frac{n}{2}}} \color{black}{\mathrm{d}x} \\&=&\frac{1}{\color{red}{2^{\frac{n}{2}}}\color{black}{\Gamma\left(\frac{n}{2}\right)}}\color{red}{2^{\frac{n}{2}}}\color{black}{\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}} \left(\frac{1}{2}\right)^{\frac{n}{2}}\mathrm{d}x} \\&=&\frac{1}{\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-\frac{x}{2}}\left(\frac{x}{2}\right)^{\frac{n}{2}}\mathrm{d}x \\&=&\frac{1}{\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-t}t^{\frac{n}{2}}\;2\mathrm{d}t \;\cdots\;t=\frac{x}{2},\frac{\mathrm{d}t}{\mathrm{d}x}=\frac{1}{2},\mathrm{d}x=2\mathrm{d}t \\&=&\frac{1}{\Gamma\left(\frac{n}{2}\right)}2\int_0^\infty e^{-t}t^{\frac{n}{2}}\;\mathrm{d}t \;\cdots\;\int cf(x) \mathrm{d}x=c\int f(x) \mathrm{d}x \\&=&\frac{1}{\Gamma\left(\frac{n}{2}\right)}2\int_0^\infty e^{-t}t^{\frac{n}{2}\color{red}{+1-1}}\;\mathrm{d}t \\&=&\frac{1}{\Gamma\left(\frac{n}{2}\right)}2\Gamma\left(\frac{n}{2}+1\right) \;\cdots\;\Gamma\left(s\right)=\int_0^\infty e^{-t}t^{s-1}\mathrm{d}t \\&=&\frac{1}{\Gamma\left(\frac{n}{2}\right)}2\frac{n}{2}\Gamma\left(\frac{n}{2}\right) \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/s1ss.html}{\Gamma(s+1)=\int_0^\infty e^{-t}t^{s}\;\mathrm{d}t=s\Gamma(s)} \\&=&n \end{eqnarray} $$カイ二乗分布の二次モーメント
$$ \begin{eqnarray} \mathrm{E}\left[x^2\right]&=&\int_0^\infty x^2 \chi^2(x) \mathrm{d}x \\&=&\int_0^\infty x^2 \frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}e^{-\frac{x}{2}}x^{\frac{n}{2}-1} \mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}\int_0^\infty x^2 e^{-\frac{x}{2}}x^{\frac{n}{2}-1} \mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}+1} \mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}+1} \color{red}{2^{\frac{n}{2}+1} \left(\frac{1}{2}\right)^{\frac{n}{2}+1} } \color{black}{\mathrm{d}x} \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}2^{\frac{n}{2}+1}\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}+1} \left(\frac{1}{2}\right)^{\frac{n}{2}+1}\mathrm{d}x \\&=&\frac{1}{2^{\frac{n}{2}}\Gamma\left(\frac{n}{2}\right)}2^{\frac{n}{2}}2\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}+1} \left(\frac{1}{2}\right)^{\frac{n}{2}+1}\mathrm{d}x \\&=&\frac{1}{\color{red}{2^{\frac{n}{2}}}\color{black}{\Gamma\left(\frac{n}{2}\right)}}\color{red}{2^{\frac{n}{2}}}\color{black}{2\int_0^\infty e^{-\frac{x}{2}}x^{\frac{n}{2}+1} \left(\frac{1}{2}\right)^{\frac{n}{2}+1}\mathrm{d}x} \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-\frac{x}{2}}\left(\frac{x}{2}\right)^{\frac{n}{2}+1}\mathrm{d}x \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}\int_0^\infty e^{-t}t^{\frac{n}{2}+1}\;2\mathrm{d}t \;\cdots\;t=\frac{x}{2},\frac{\mathrm{d}t}{\mathrm{d}x}=\frac{1}{2},\mathrm{d}x=2\mathrm{d}t \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}2\int_0^\infty e^{-t}t^{\frac{n}{2}+1}\;\mathrm{d}t \;\cdots\;\int cf(x) \mathrm{d}x=c\int f(x) \mathrm{d}x \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}2\int_0^\infty e^{-t}t^{\frac{n}{2}+1\color{red}{+1-1}}\;\mathrm{d}t \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}2\int_0^\infty e^{-t}t^{\frac{n}{2}+2-1}\;\mathrm{d}t \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}2\Gamma\left(\frac{n}{2}+2\right) \;\cdots\;\Gamma\left(s\right)=\int_0^\infty e^{-t}t^{s-1}\mathrm{d}t \\&=&\frac{2}{\Gamma\left(\frac{n}{2}\right)}2\left(\frac{n}{2}+1\right)\frac{n}{2}\Gamma\left(\frac{n}{2}\right) \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/08/s1ss.html}{\Gamma\left(s+2\right)=(s+1)\Gamma(s+1)=(s+1)s\Gamma(s)} \\&=&2n\left(\frac{n}{2}+1\right) \\&=&n(n+2) \end{eqnarray} $$カイ二乗分布の分散(二次の中心モーメント)
$$ \begin{eqnarray} \mathrm{V}\left[x^2\right]&=&\mathrm{E}\left[(x-\mathrm{E}\left[x\right])^2\right] \\&=&\mathrm{E}\left[x^2\right]-\mathrm{E}\left[x\right]^2 \\&=&n(n+2)-n^2 \\&=&n^2+2n-n^2 \\&=&2n \end{eqnarray} $$ベータ分布の期待値(平均)と分散
$$
\begin{eqnarray}
f(x;m,n)&=&\href{https://shikitenkai.blogspot.com/2020/05/blog-post_22.html}{\frac{x^{(m-1)}(1-x)^{(n-1)}}{B(m,n)}\;\cdots\;ベータ分布}
\\B(m,n)&=&\int_0^1x^{(m-1)}(1-x)^{(n-1)} \mathrm{d}x\;\cdots\;ベータ凾数
\\&=&\frac{(m-1)!\;(n-1)!}{\left\{(m-1)+(n-1)+1\right\}!}
\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/05/blog-post_22.html}{\int_\alpha^\beta(x-\alpha)^p(\beta-x)^q \mathrm{d}x=\frac{p!\;q!}{(p+q+1)!}(\beta-\alpha)^{(p+q+1)}(第一種オイラー積分)}
\\&=&\frac{(m-1)!\;(n-1)!}{(m+n-1)!}
\end{eqnarray}
$$
ベータ分布の期待値(一次モーメント・平均)
$$ \begin{eqnarray} \mathbf{E}[X]&=&\int_0^1xf(x)\mathrm{d}x \\&=&\int_0^1x\; \frac{x^{(m-1)}(1-x)^{(n-1)}}{B(m,n)} \;\mathrm{d}x \\&=&\frac{1}{B(m,n)} \int_0^1x^{m}(1-x)^{(n-1)}\;\mathrm{d}x \\&=&\frac{(m+n-1)!}{(m-1)!\;(n-1)!}\frac{m!\;(n-1)!}{(m+n)!} \\&=&\frac{(m+n-1)!}{(m+n)!}\frac{m!}{(m-1)!} \\&=&\frac{1}{m+n}\frac{m}{1} \\&=&\frac{m}{m+n}\;\cdots\;ベータ分布の平均 \end{eqnarray} $$ベータ分布の二次モーメント
$$ \begin{eqnarray} \mathbf{E}[X^2]&=&\int_0^1x^2f(x)\mathrm{d}x \\&=&\int_0^1x^2\; \frac{x^{(m-1)}(1-x)^{(n-1)}}{B(m,n)} \;\mathrm{d}x \\&=&\frac{1}{B(m,n)} \int_0^1x^{(m+1)}(1-x)^{(n-1)}\;\mathrm{d}x \\&=&\frac{(m+n-1)!}{(m-1)!\;(n-1)!}\frac{(m+1)!\;(n-1)!}{((m+1)+n)!} \\&=&\frac{(m+n-1)!}{(m-1)!\;(n-1)!}\frac{(m+1)!\;(n-1)!}{(m+n+1)!} \\&=&\frac{(m+n-1)!}{(m+n+1)!}\frac{(m+1)!}{(m-1)!} \\&=&\frac{1}{(m+n+1)(m+n)}\frac{(m+1)m}{1} \\&=&\frac{(m+1)m}{(m+n+1)(m+n)} \end{eqnarray} $$ベータ分布の分散(二次の中心モーメント
$$ \begin{eqnarray} \mathbf{V}[X]&=&\mathbf{E}[X^2]-\mathbf{E}[X]^2 \\&=&\frac{(m+1)m}{(m+n+1)(m+n)}-\left\{\frac{m}{m+n}\right\}^2 \\&=&\frac{(m+1)m}{(m+n+1)(m+n)}-\frac{m^2}{\left(m+n\right)^2} \\&=&\frac{(m+n)(m+1)m-(m+n+1)m^2}{(m+n+1)(m+n)^2} \\&=&\frac{(m^2+m+mn+n)m-(m^3+m^2n+m^2)}{(m+n+1)(m+n)^2} \\&=&\frac{m^3+m^2+m^2n+mn-m^3-m^2n-m^2}{(m+n+1)(m+n)^2} \\&=&\frac{mn}{(m+n+1)(m+n)^2}\;\cdots\;ベータ分布の分散 \end{eqnarray} $$スコア凾数の期待値と分散
$$
\href{https://shikitenkai.blogspot.com/2020/04/blog-post.html}{\frac{\partial \log{f(x;\theta)}}{\partial \theta}:スコア凾数}\\
$$
スコア凾数の期待値
$$ \begin{eqnarray} \mathrm{E}\left[ \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]\ &=&\int{ \frac{\partial \log{f(x;\theta)}}{\partial \theta} f(x;\theta)\mathrm{d}x}\\ &=&\int{ \frac{1}{f(x;\theta)} \frac{\partial f(x;\theta)}{\partial \theta} f(x;\theta)\mathrm{d}x} \;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}\log{f(x)}=\frac{1}{f(x)}\frac{\mathrm{d}f(x)}{\mathrm{d}x}\\ &=&\int{ \frac{\partial f(x;\theta)}{\partial \theta} \mathrm{d}x}\\ &=&\frac{\partial }{\partial \theta} \int{f(x;\theta) \mathrm{d}x}\\ &=&\frac{\partial }{\partial \theta} 1\;\cdots\;\int{f(x;\theta) \mathrm{d}x}=1\\ &=&0\;\cdots\;定数の微分は0\\ \end{eqnarray} $$スコア凾数の分散
$$ \mathcal{I}=\mathrm{V}\left[\frac{ \partial \log{ f(x;\theta) }}{ \partial \theta }\right]:フィッシャー情報量\\ $$不偏分散の期待値
不偏分散の期待値
$$\begin{array}{rclcl} \hat{\sigma}^2 &=& \href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{\displaystyle \frac{1}{n-1}\sum_{k=1}^{n}\left( \displaystyle X_k - \overline{X} \displaystyle \right)^2}\,\dotso\,不偏分散(unbiased \, variance)\\ E\left[\hat{\sigma}^2\right] &=&E\left[ \displaystyle\frac{1}{n-1}\sum_{k=1}^{n} \left(X_k -\overline{X}\right)^2 \right]\\ &=&\displaystyle\frac{1}{n-1}E\left[\sum_{k=1}^{n} \left(X_k -\overline{X}\right)^2 \right]\\ &=&\displaystyle\frac{1}{n-1}\left(n-1\right)\sigma^2 \,\dotso\,\displaystyle \href{https://shikitenkai.blogspot.com/2019/07/overlinex2.html}{E\left[\sum_{k=1}^{n} \left(X_k -\overline{X}\right)^2 \right]=\left(n-1\right)\sigma^2}\\ &=&\displaystyle\sigma^2\\ \end{array}$$標本分散の期待値
$$\begin{array}{rcl}
s^2&=&\displaystyle \frac{1}{n}\sum_{k=1}^{n}\left(
\displaystyle X_k - \overline{X}
\displaystyle \right)^2\,\dotso\,標本分散(sample \, variance)\\
E\left[s^2\right]&=&E\left[ \displaystyle\frac{1}{n}\sum_{k=1}^{n} \left(X_k -\overline{X}\right)^2 \right]\\
&=&\displaystyle \frac{1}{n} E\left[ \sum_{k=1}^{n} \left(X_k -\overline{X}\right)^2 \right]\\
&=&\displaystyle \frac{1}{n} \left(n-1\right)\sigma^2 \,\dotso\,\displaystyle \href{https://shikitenkai.blogspot.com/2019/07/overlinex2.html}{E\left[\sum_{k=1}^{n} \left(X_k -\overline{X}\right)^2 \right]=\left(n-1\right)\sigma^2}\\
&=&\displaystyle \frac{n-1}{n}\sigma^2 \\
\end{array}$$
連続型確率変数(continuous random variable) の一様分布(uniform distribution)の期待値(expected value)
$$\begin{array}{rcl}
\displaystyle M_X^{(m)}(0)&\equiv&\frac{ \mathrm{d}^m }{ \mathrm{d}^m t } M_X(t)|_{t=0}\\
&=&\displaystyle E[X^m\mathrm{e}^{tX}]|_{t=0}\\
&=&\displaystyle E[X^m]\\
\end{array}$$
積率母凾数の一階微分
$$\begin{array}{rcl} \displaystyle M_X^{(1)} &=& \displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\left\{ \displaystyle \href{https://shikitenkai.blogspot.com/2019/07/continuous-random-variable-uniform.html}{\frac{\mathrm{e}^{tb}-\mathrm{e}^{ta}}{t(b-a)}} \displaystyle \right\}\\ &=& \displaystyle \frac{1}{b-a} \displaystyle \left\{ \displaystyle \left(t^{-1}\right)'\left(\mathrm{e}^{tb}-\mathrm{e}^{ta}\right) \displaystyle +\left(t^{-1}\right)\left(\mathrm{e}^{tb}-\mathrm{e}^{ta}\right)' \displaystyle \right\}\\ &=& \displaystyle \frac{1}{b-a} \displaystyle \left\{ \displaystyle \left(-t^{-2}\right)\left(\mathrm{e}^{tb}-\mathrm{e}^{ta}\right) \displaystyle +\left(t^{-1}\right)\left(b\mathrm{e}^{tb}-a\mathrm{e}^{ta}\right) \displaystyle \right\}\\ &=& \displaystyle \frac{1}{b-a} \displaystyle \left( \displaystyle -\frac{\mathrm{e}^{tb}-\mathrm{e}^{ta}}{t^2} \displaystyle +\frac{b\mathrm{e}^{tb}-a\mathrm{e}^{ta}}{t} \displaystyle \right)\\ &=& \displaystyle \frac{1}{b-a} \displaystyle \left\{ \displaystyle -\frac{\mathrm{e}^{tb}-\mathrm{e}^{ta}}{t^2} \displaystyle +\frac{(b\mathrm{e}^{tb}-a\mathrm{e}^{ta})t}{t^2} \displaystyle \right\}\\ &=& \displaystyle \frac{1}{b-a} \displaystyle \left( \displaystyle \frac{tb\mathrm{e}^{tb}-\mathrm{e}^{tb}-ta\mathrm{e}^{ta}+\mathrm{e}^{ta}}{t^2} \displaystyle \right)\\ &=& \displaystyle \frac{1}{t^2(b-a)} \displaystyle \left\{ \displaystyle (tb-1)\mathrm{e}^{tb}-(ta-1)\mathrm{e}^{ta} \displaystyle \right\}\\ &=& \displaystyle \frac{(tb-1)\mathrm{e}^{tb}-(ta-1)\mathrm{e}^{ta}}{t^2(b-a)}\\ \end{array}$$原点周りの一次モーメント=期待値
$$\begin{array}{rcl} \displaystyle E[X]&=&\displaystyle M_X^{(1)}(0)\\ &=&\displaystyle \lim_{t \to 0}\left\{ \displaystyle \frac{(tb-1)\mathrm{e}^{tb}-(ta-1)\mathrm{e}^{ta}}{t^2(b-a)} \displaystyle \right\}\,\dotso\,0を代入すると分母が0になってしまうので極限で考える．\\ &=&\displaystyle \lim_{t \to 0}\left[ \displaystyle \frac{1}{t^2(b-a)}\left\{ \displaystyle (tb-1)\mathrm{e}^{tb}-(ta-1)\mathrm{e}^{ta} \displaystyle \right\} \displaystyle \right]\\ &=&\displaystyle \lim_{t \to 0}\left[ \displaystyle \frac{1}{t^2(b-a)}\left\{ \displaystyle (tb-1)\left(\frac{(tb)^0}{0!}+\frac{(tb)^1}{1!}+\frac{(tb)^2}{2!}\right) \displaystyle -(ta-1)\left(\frac{(ta)^0}{0!}+\frac{(ta)^1}{1!}+\frac{(ta)^2}{2!}\right) \displaystyle \right\} \displaystyle \right]\\ && \,\dotso\,\href{https://shikitenkai.blogspot.com/2019/07/blog-post.html}{\mathrm{e}^x=\sum_{k=0}^{\infty}\frac{x^k}{k!}=\frac{x^0}{0!}+\frac{x^1}{1!}+\frac{x^2}{2!}+\dotsb} (マクローリン展開), t^2が分母にあるのでt^3の項以上は分子にtが残ることになるのでt^2の項までで計算を進める．\\ &=&\displaystyle \lim_{t \to 0}\left[ \displaystyle \frac{1}{t^2(b-a)}\left\{ \displaystyle (tb-1)\left(1+tb+t^2\frac{b^2}{2}\right) \displaystyle -(ta-1)\left(1+ta+t^2\frac{a^2}{2}\right) \displaystyle \right\} \displaystyle \right]\\ &=&\displaystyle \lim_{t \to 0}\left[ \displaystyle \frac{1}{t^2(b-a)}\left[ \displaystyle \left\{\left(tb+t^2b^2+t^3\frac{b^3}{2}\right)-\left(1+tb+t^2\frac{b^2}{2}\right)\right\} \displaystyle -\left\{\left(ta+t^2a^2+t^3\frac{a^3}{2}\right)-\left(1+ta+t^2\frac{a^2}{2}\right)\right\} \displaystyle \right] \displaystyle \right]\\ &=&\displaystyle \lim_{t \to 0}\left[ \displaystyle \frac{1}{t^2(b-a)}\left\{ \displaystyle \left(-1+t(b-b)+t^2(b^2-\frac{b^2}{2})+t^3\frac{b^3}{2}\right) \displaystyle -\left(-1+t(a-a)+t^2(a^2-\frac{a^2}{2})+t^3\frac{a^3}{2}\right) \displaystyle \right\} \displaystyle \right]\\ &=&\displaystyle \lim_{t \to 0}\left[ \displaystyle \frac{1}{t^2(b-a)}\left\{ \displaystyle \left(-1+t^2\frac{b^2}{2}+t^3\frac{b^3}{2}\right) \displaystyle -\left(-1+t^2\frac{a^2}{2}+t^3\frac{a^3}{2}\right) \displaystyle \right\} \displaystyle \right]\\ &=&\displaystyle \lim_{t \to 0}\left\{ \displaystyle \frac{1}{t^2(b-a)}\left( \displaystyle t^2\frac{b^2-a^2}{2} \displaystyle +t^3\frac{b^3-a^3}{2} \displaystyle \right) \displaystyle \right\}\\ &=&\displaystyle \lim_{t \to 0} \left[\frac{1}{t^2(b-a)}\left\{ \displaystyle t^2 \frac{ (b-a)(b+a) }{2} \displaystyle +t^3 \frac{ (b-a)(b^2+ab+a^2) }{2} \displaystyle \right\}\right] \,\dotso\,b^2-a^2=(b-a)(b+a),\,b^3-a^3=(b-a)(b^2+ab+a^2)\\ &=&\displaystyle \lim_{t \to 0} \displaystyle \left\{ \displaystyle \frac{ (b+a) }{2} \displaystyle +t \frac{ (b^2+ab+a^2) }{2} \displaystyle \right\}\\ &=&\displaystyle \frac{b+a}{2}=\frac{a+b}{2} \,\dotso\,tが分子にある(掛けられている)項は全て0．\\ \end{array}$$離散型確率変数(discrete random variable) の一様分布(uniform distribution)の期待値(expected value)
$$\begin{array}{rcl}
\displaystyle M_X^{(m)}(0)&\equiv&\frac{ \mathrm{d}^m }{ \mathrm{d}^m t } M_X(t)|_{t=0}\\
&=&\displaystyle E[X^m\mathrm{e}^{tX}]|_{t=0}\\
&=&\displaystyle E[X^m]\\
\end{array}$$
積率母凾数の一階微分
$$\begin{array}{rcl} \displaystyle M_X^{(1)} &=& \displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\left\{ \displaystyle \href{https://shikitenkai.blogspot.com/2019/07/uniform-distribution.html}{\frac{1}{n}\frac{\mathrm{e}^{t}(\mathrm{e}^{nt}-1)}{(\mathrm{e}^{t}-1)}} \displaystyle \right\}\\ &=& \displaystyle \frac{1}{n} \displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\left\{ \displaystyle \frac{\mathrm{e}^{t}(\mathrm{e}^{nt}-1)}{(\mathrm{e}^{t}-1)} \displaystyle \right\}\\ &=& \displaystyle \frac{1}{n} \displaystyle \frac{\mathrm{d}}{\mathrm{d}t}\left\{ \displaystyle \mathrm{e}^{t}(\mathrm{e}^{nt}-1)(\mathrm{e}^{t}-1)^{-1} \displaystyle \right\}\\ &=&\displaystyle \frac{1}{n} \left\{ (\mathrm{e}^{t})'(\mathrm{e}^{nt}-1)(\mathrm{e}^{t}-1)^{-1} + \mathrm{e}^{t}(\mathrm{e}^{nt}-1)'(\mathrm{e}^{t}-1)^{-1} + \mathrm{e}^{t}(\mathrm{e}^{nt}-1)((\mathrm{e}^{t}-1)^{-1})' \right\}\\ &&\,\dotso\,(uvw)'=u'(vw)+u(vw)'=u'(vw)+u(v'w+vw')=u'vw+uv'w+uvw'\\ &=&\displaystyle \frac{1}{n} \left\{ \mathrm{e}^{t}(\mathrm{e}^{nt}-1)(\mathrm{e}^{t}-1)^{-1} + \mathrm{e}^{t}(n\mathrm{e}^{nt})(\mathrm{e}^{t}-1)^{-1} + \mathrm{e}^{t}(\mathrm{e}^{nt}-1)(-\mathrm{e}^{t}(\mathrm{e}^{t}-1)^{-2}) \right\}\\ &=&\displaystyle \frac{1}{n} \left\{ \frac{\mathrm{e}^{t}(\mathrm{e}^{nt}-1)}{(\mathrm{e}^{t}-1)} + \frac{\mathrm{e}^{t}(n\mathrm{e}^{nt})}{(\mathrm{e}^{t}-1)} + \frac{\mathrm{e}^{t}\mathrm{e}^{t}(\mathrm{e}^{nt}-1)}{-(\mathrm{e}^{t}-1)^2} \right\}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \left\{ \frac{(\mathrm{e}^{nt}-1)}{(\mathrm{e}^{t}-1)} + \frac{(n\mathrm{e}^{nt})}{(\mathrm{e}^{t}-1)} + \frac{\mathrm{e}^{t}(\mathrm{e}^{nt}-1)}{-(\mathrm{e}^{t}-1)^2} \right\}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \frac{(\mathrm{e}^{nt}-1)(\mathrm{e}^{t}-1) + (n\mathrm{e}^{nt})(\mathrm{e}^{t}-1) - \mathrm{e}^{t}(\mathrm{e}^{nt}-1)}{(\mathrm{e}^{t}-1)^2}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \frac{(\mathrm{e}^{nt}\mathrm{e}^{t}-\mathrm{e}^{nt}-\mathrm{e}^{t}+1) + (n\mathrm{e}^{nt}\mathrm{e}^{t}-n\mathrm{e}^{nt}) - (\mathrm{e}^{nt}\mathrm{e}^{t}-\mathrm{e}^{t})}{(\mathrm{e}^{t}-1)^2}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \frac{\mathrm{e}^{nt}\mathrm{e}^{t}-\mathrm{e}^{nt}-\mathrm{e}^{t}+1 + n\mathrm{e}^{nt}\mathrm{e}^{t}-n\mathrm{e}^{nt} - \mathrm{e}^{nt}\mathrm{e}^{t}+\mathrm{e}^{t}}{(\mathrm{e}^{t}-1)^2}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \frac{\mathrm{e}^{nt}(\mathrm{e}^{t} - 1 +n\mathrm{e}^{t} -n-\mathrm{e}^{t}) +1}{(\mathrm{e}^{t}-1)^2}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \frac{\mathrm{e}^{nt}(n\mathrm{e}^{t}-(n+1))+1}{(\mathrm{e}^{t}-1)^2}\\ &=&\displaystyle \frac{\mathrm{e}^{t}}{n} \frac{n\mathrm{e}^{(n+1)t}-(n+1)\mathrm{e}^{nt}+1}{(\mathrm{e}^{t}-1)^2}\\ \end{array}$$原点周りの一次モーメント=期待値
$$\begin{array}{rcl} \displaystyle E[X]&=&\displaystyle M_X^{(1)}(0)\\ &=&\displaystyle \lim_{t \to 0}\left\{ \frac{\mathrm{e}^{t}}{n} \displaystyle \frac{n\mathrm{e}^{(n+1)t}-(n+1)\mathrm{e}^{nt}+1}{(\mathrm{e}^{t}-1)^2} \right\}\,\dotso\,0を代入すると分母が0になってしまうので極限で考える．\\ &=&\displaystyle \lim_{t \to 0}\left\{ \frac{\mathrm{e}^{t}}{n} \displaystyle \frac{n\mathrm{e}^{(n+1)t}-(n+1)\mathrm{e}^{nt}+1}{\mathrm{e}^{2t}-2\mathrm{e}^{t}+1} \right\}\\ &=&\displaystyle \lim_{t \to 0} \left\{\frac{\left(\frac{t^0}{0!}+\frac{t^1}{1!}+\frac{t^2}{2!}+\frac{t^3}{3!}\right)}{n} \frac{n\left(\frac{((n+1)t)^0}{0!}+\frac{((n+1)t)^1}{1!}+\frac{((n+1)t)^2}{2!}+\frac{((n+1)t)^3}{3!}\right) -(n+1)\left(\frac{(nt)^0}{0!}+\frac{(nt)^1}{1!}+\frac{(nt)^2}{2!}+\frac{(nt)^3}{3!}\right) +1}{ \left(\frac{(2t)^0}{0!}+\frac{(2t)^1}{1!}+\frac{(2t)^2}{2!}+\frac{(2t)^3}{3!}\right) -2 \left(\frac{t^0}{0!}+\frac{t^1}{1!}+\frac{t^2}{2!}+\frac{t^3}{3!}\right) +1} \right\}\\ && \displaystyle \,\dotso\,\href{https://shikitenkai.blogspot.com/2019/07/blog-post.html}{\mathrm{e}^x=\sum_{k=0}^{\infty}\frac{x^k}{k!}=\frac{x^0}{0!}+\frac{x^1}{1!}+\frac{x^2}{2!}+\frac{x^3}{3!}+\dotsb} (マクローリン展開), ひとまずt^3の項までで計算を進める．\\ &=&\displaystyle \lim_{t \to 0}\left\{ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{n} \frac{n\left(1+(n+1)t+\frac{(n+1)^2}{2}t^2+\frac{(n+1)^3}{6}t^3\right) -(n+1)\left(1+nt+\frac{n^2}{2}t^2+\frac{n^3}{6}t^3\right) +1}{ \left(1+(2t)+\frac{(2t)^2}{2}+\frac{(2t)^3}{6}\right) -2 \left(1+(t)+\frac{t^2}{2}+\frac{t^3}{6}\right) +1} \right\}\\ &=&\displaystyle \lim_{t \to 0}\left\{ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{n} \frac{n\left(1+(n+1)t+\frac{(n+1)^2}{2}t^2+\frac{(n+1)^3}{6}t^3\right) -(n+1)\left(1+nt+\frac{n^2}{2}t^2+\frac{n^3}{6}t^3\right) +1}{ 1+2t+2t^2+\frac{4}{3}t^3 -2-2t-t^2-\frac{1}{3}t^3 +1} \right\}\\ &=&\displaystyle \lim_{t \to 0}\left\{ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{n} \frac{n\left(1+(n+1)t+\frac{(n+1)^2}{2}t^2+\frac{(n+1)^3}{6}t^3\right) -(n+1)\left(1+nt+\frac{n^2}{2}t^2+\frac{n^3}{6}t^3\right) +1}{t^2(1+t)} \right\}\\ &=&\displaystyle \lim_{t \to 0}\left[ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{nt^2(1+t)} \left\{n+n(n+1)t+\frac{n(n+1)^2}{2}t^2+\frac{n(n+1)^3}{6}t^3 -(n+1)-(n+1)nt-\frac{(n+1)n^2}{2}t^2-\frac{(n+1)n^3}{6}t^3 +1\right\} \right]\\ &=&\displaystyle \lim_{t \to 0}\left[ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{nt^2(1+t)} \left\{(n-(n+1)+1) +(n(n+1)-(n+1)n)t +(\frac{n(n+1)^2}{2}-\frac{(n+1)n^2}{2})t^2 +(\frac{n(n+1)^3}{6}-\frac{(n+1)n^3}{6})t^3 \right\} \right]\\ &=&\displaystyle \lim_{t \to 0}\left[ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{nt^2(1+t)} \left\{(0) +(0)t +(\frac{n+1}{2})nt^2 +(\frac{(n+1)(2n+1)}{6})nt^3 \right\} \right]\\ &=&\displaystyle \lim_{t \to 0}\left\{ \frac{\left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right)}{nt^2(1+t)} nt^2\left(\frac{n+1}{2}+\frac{(n+1)(2n+1)}{6}t\right) \right\}\\ &&\,\dotso\,分母はt^2の項からが残っている．t^3以上の項はtが残るのでマクローリン展開はt^3で十分となる．\\ &=&\displaystyle \lim_{t \to 0}\left\{ \left(1+t+\frac{1}{2}t^2+\frac{1}{6}t^3\right) \left(\frac{n+1}{2}+\frac{(n+1)(2n+1)}{6}t\right) \right\}\\ &=&\displaystyle \frac{n+1}{2}\\ &&\,\dotso\,tが分子にある(掛けられている)項は全て0．\\ \end{array}$$正規分布(normal distribution)の積率母凾数(moment-generating function)と期待値(expected value)・分散(variance)
正規分布
$$\begin{array}{rcl} N(\mu, \sigma^2)&=&\frac{1}{\sqrt{2\pi \sigma^2}}\mathrm{e}^{\frac{-(x-\mu)^2}{2\sigma^2}} \end{array}$$積率母凾数
$$\begin{array}{rcl} \displaystyle M_X(t)&\equiv&\displaystyle E[\mathrm{e}^{tX}]\\ &=&\displaystyle \int_{-\infty}^{\infty}(\mathrm{e}^{tx})\frac{1}{\sqrt{2\pi \sigma^2}}\mathrm{e}^{-\frac{(x-\mu)^2}{2\sigma^2}} \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty}(\mathrm{e}^{tx})\mathrm{e}^{-\frac{(x-\mu)^2}{2\sigma^2}} \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty}\mathrm{e}^{-\frac{(x-\mu)^2}{2\sigma^2}+tx} \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty}\mathrm{e}^{-\frac{(x-\mu)^2+(2\sigma^2)(tx)}{2\sigma^2}} \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty}\mathrm{e}^{-\frac{x^2-2x\mu+\mu^2-2x\sigma^2t}{2\sigma^2}} \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty} \displaystyle \mathrm{e}^{-\frac{x^2-2x\mu+\mu^2-2x\sigma^2t}{2\sigma^2}-\frac{2\mu\sigma^2 t+\sigma^4t^2}{2\sigma^2}+\frac{2\mu\sigma^2 t+\sigma^4t^2}{2\sigma^2}} \displaystyle \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty} \displaystyle \mathrm{e}^{-\frac{x^2-2x\mu+\mu^2-2x\sigma^2t+2\mu\sigma^2 t+\sigma^4t^2}{2\sigma^2}+\frac{2\mu\sigma^2 t+\sigma^4t^2}{2\sigma^2}} \displaystyle \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty} \displaystyle \mathrm{e}^{-\frac{(x-\mu-\sigma^2t)^2}{2\sigma^2}+\frac{2\mu\sigma^2 t+\sigma^4t^2}{2\sigma^2}} \displaystyle \mathrm{d}x \,\dotso\,a^2-2ab+b^2-2ac+2bc+c^2=(a-b-c)^2\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty} \displaystyle \mathrm{e}^{-\frac{(x-\mu-\sigma^2t)^2}{2\sigma^2}+(\mu t + \frac{\sigma^2t^2}{2})} \displaystyle \mathrm{d}x\\ &=&\displaystyle \frac{1}{\sqrt{2\pi \sigma^2}} \displaystyle \int_{-\infty}^{\infty} \mathrm{e}^{-\frac{(x-\mu-\sigma^2t)^2}{2\sigma^2}} \displaystyle \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})} \displaystyle \mathrm{d}x\\ &=&\displaystyle \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})} \displaystyle \frac{1}{\sqrt{2\pi \sigma^2}} \displaystyle \int_{-\infty}^{\infty}\mathrm{e}^{-\frac{(x-\mu-\sigma^2t)^2}{2\sigma^2}}\mathrm{d}x \\ &=&\displaystyle \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})}\,\dotso\,\frac{1}{\sqrt{2\pi \sigma^2}}\int_{-\infty}^{\infty}\mathrm{e}^{-\frac{(x-\mu-\sigma^2t)^2}{2\sigma^2}}\mathrm{d}x = N(\mu+\sigma^2t,\sigma^2)の総和=1\\ \end{array}$$期待値・分散
$$\begin{array}{rcl} \displaystyle M_X^{(m)}(0)&\equiv&\frac{ \mathrm{d}^m }{ \mathrm{d}^m t } M_x(t)|_{t=0}\\ &=&\displaystyle E[X^m\mathrm{e}^{tX}]|_{t=0}\\ &=&\displaystyle E[X^m]\\ \end{array}$$ $$\begin{array}{rcl} \displaystyle E[X]&=&\displaystyle M_X^{(1)}(0)\\ &=&\displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} t }\left(\mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})}\right) \right\}|_{t=0}\\ &=&\displaystyle \left\{ \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})}(\mu+\sigma^2t) \right\}|_{t=0}\\ &=&\displaystyle \mathrm{e}^{(\mu 0 + \frac{\sigma^20^2}{2})}(\mu+\sigma^20) \\ &=&\displaystyle \mathrm{e}^{0}(\mu+0) \\ &=&\displaystyle \mu \\ \end{array}$$ $$\begin{array}{rcl} \displaystyle E[X^2]&=&\displaystyle M_X^{(2)}(0)\\ &=&\displaystyle \left\{ \frac{ \mathrm{d}^2 }{ \mathrm{d} t^2 }\left(\mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})}\right) \right\}|_{t=0}\\ &=&\displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} t }\left( \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})}(\mu+\sigma^2t)\right) \right\}|_{t=0}\\ &=&\displaystyle \left[ \displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} t } \left( \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})} \right)\right\} \left( \mu+\sigma^2t \right) \displaystyle + \left( \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})} \right) \left\{ \frac{ \mathrm{d} }{ \mathrm{d} t } \left( \mu+\sigma^2t \right) \right\} \right]|_{t=0}\\ &=&\displaystyle \left[ \displaystyle \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})} \left( \mu+\sigma^2t \right)^2 \displaystyle + \mathrm{e}^{(\mu t + \frac{\sigma^2t^2}{2})} \sigma^2 \displaystyle \right]|_{t=0}\\ &=&\displaystyle \mathrm{e}^{(\mu 0 + \frac{\sigma^20^2}{2})} \left( \mu+\sigma^20 \right)^2 \displaystyle + \mathrm{e}^{(\mu 0 + \frac{\sigma^20^2}{2})} \sigma^2\\ &=&\displaystyle \mu^2+\sigma^2\\ \end{array}$$ $$\begin{array}{rcl} V[X]&=&E[X^2]-E[X]^2\\ &=&\mu^2+\sigma^2-\mu^2\\ &=&\sigma^2 \end{array}$$ポアソン分布(Poisson distribution)の積率母凾数(moment-generating function)と期待値(expected value)・分散(variance)
ポアソン分布
$$\begin{array}{rcl} \displaystyle Po(\lambda)&=&\displaystyle \frac{\lambda^{x}}{x!}\mathrm{e}^{-\lambda}\\ \end{array}$$積率母凾数
$$\begin{array}{rcl} \displaystyle M_X(t)&\equiv&\displaystyle E[\mathrm{e}^{tX}]\\ &=&\displaystyle \sum_{x=0}^{\infty}(\mathrm{e}^{tx})\frac{\lambda^{x}}{x!}\mathrm{e}^{-\lambda}\\ &=&\displaystyle \mathrm{e}^{-\lambda}\sum_{x=0}^{\infty}(\mathrm{e}^{tx})\frac{\lambda^{x}}{x!}\\ &=&\displaystyle \mathrm{e}^{-\lambda}\sum_{x=0}^{\infty}\frac{\mathrm{e}^{tx}\lambda^{x}}{x!}\\ &=&\displaystyle \mathrm{e}^{-\lambda}\sum_{x=0}^{\infty}\frac{(\mathrm{e}^{t}\lambda)^{x}}{x!}\\ &=&\displaystyle \mathrm{e}^{-\lambda}\mathrm{e}^{\mathrm{e}^{t}\lambda} \,\dotso\,\href{https://shikitenkai.blogspot.com/2019/07/blog-post.html}{\sum_{x=0}^{\infty}\frac{a^{x}}{x!}=\mathrm{e}^a}\\ &=&\displaystyle \mathrm{e}^{\mathrm{e}^{t}\lambda-\lambda}=\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)} \end{array}$$期待値・分散
$$\begin{array}{rcl} \displaystyle M_X^{(m)}(0)&\equiv&\frac{ \mathrm{d}^m }{ \mathrm{d}^m t } M_x(t)|_{t=0}\\ &=&\displaystyle E[X^m\mathrm{e}^{tX}]|_{t=0}\\ &=&\displaystyle E[X^m]\\ \end{array}$$ $$\begin{array}{rcl} \displaystyle E[X]&=&\displaystyle M_X^{(1)}(0)\\ &=&\displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} t }(\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)}) \right\}|_{t=0}\\ &=&\displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} s }(\mathrm{e}^{s})\frac{ \mathrm{d}s}{ \mathrm{d}t} \right\}|_{t=0} \,\dotso\,s=\lambda(\mathrm{e}^{t}-1),\frac{ \mathrm{d}s}{ \mathrm{d}t}=\lambda\mathrm{e}^{t}\\ &=&\displaystyle \left\{ (\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)})(\lambda\mathrm{e}^{t}) \right\}|_{t=0} \,\dotso\,\frac{ \mathrm{d} }{ \mathrm{d} x }\mathrm{e}^{x}=\mathrm{e}^{x}\\ &=&\displaystyle \left\{ \lambda(\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)+t}) \right\}|_{t=0}\\ &=&\displaystyle \lambda(\mathrm{e}^{\lambda(\mathrm{e}^{0}-1)+0})\\ &=&\displaystyle \lambda\mathrm{e}^0 \,\dotso\,a^0=1\\ &=&\lambda \,\dotso\,a^0=1\\ \end{array}$$ $$\begin{array}{rcl} \displaystyle E[X^2]&=&\displaystyle M_X^{(2)}(0)\\ &=&\displaystyle \left\{ \frac{ \mathrm{d}^2 }{ \mathrm{d} t^2 }(\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)}) \right\}|_{t=0}\\ &=&\displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} t } \lambda(\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)+t}) \right\}|_{t=0} \,\dotso\,E[X]の展開から．\\ &=&\displaystyle \left\{ \frac{ \mathrm{d} }{ \mathrm{d} s }(\lambda\mathrm{e}^{s})\frac{ \mathrm{d}s}{ \mathrm{d}t} \right\}|_{t=0} \,\dotso\,s=\lambda(\mathrm{e}^{t}-1)+t,\frac{ \mathrm{d}s}{ \mathrm{d}t}=\lambda\mathrm{e}^{t}+1\\ &=&\displaystyle \left\{ (\lambda\mathrm{e}^{\lambda(\mathrm{e}^{t}-1)+1})(\lambda\mathrm{e}^{t}+1) \right\}|_{t=0} \,\dotso\,\frac{ \mathrm{d} }{ \mathrm{d} x }C\mathrm{e}^{x}=C\mathrm{e}^{x}\\ &=&\displaystyle (\lambda\mathrm{e}^{\lambda(\mathrm{e}^{0}-1)+1})(\lambda\mathrm{e}^{0}+1)\\ &=&\displaystyle \lambda\mathrm{e}^0(\lambda+1) \,\dotso\,a^0=1\\ &=&\lambda(\lambda+1) \,\dotso\,a^0=1\\ \end{array}$$ $$\begin{array}{rcl} \displaystyle V[X]&=&\displaystyle E[X^2]-E[X]^2\\ &=&\lambda(\lambda+1)-\lambda^2\\ &=&\lambda^2+\lambda-\lambda^2\\ &=&\lambda\\ \end{array}$$標本確率変数(Specimen random variable) / 標本確率変数の期待値と分散
$$\begin{array}{rcl}
母集団の確率変数&:&X\\
標本確率変数&:&X_k (k=1,2,\dotsc ,n)\\
\end{array}$$
$$\begin{array}{rcl}
\mu&=&E[X]\\
\sigma^2&=&V[X]\\
\end{array}$$
標本確率変数\(X_k\)は母集団の確率変数\(X\)と同じ確率分布(母集団)に従うので，\(X_k\)の期待値・分散は\(X\)の期待値・分散と等しい．
$$\begin{array}{rclcl}
E[X_k]&=&E[X]&=&\mu\\
V[X_k]&=&V[X]&=&\sigma^2\\
\end{array}$$
二項分布の期待値
二項分布 \(B(n, p)\)
$$B(n, p) = f_X(x) = \begin{cases} \displaystyle _nC_x\,p^x(1-p)^{n-x} & \quad x \in \left\{0,1,2, \dotsc ,n\right\}\\ \displaystyle 0 & \quad x \notin \left\{0,1,2, \dotsc ,n\right\} \end{cases} $$二項分布の期待値
$$\begin{array}{rcl} E[X]&=&\displaystyle \sum_{x=0}^{n}\left(x\right)\left( _nC_x\,p^x(1-p)^{n-x} \right)\\ &=& \displaystyle \sum_{x=0}^{n}\left(x\right)\left(\left( \frac{n!}{x!(n-x)!}\right)p^x(1-p)^{n-x}\right)\\ &=& \displaystyle \sum_{x=0}^{n} \frac{n!}{(x-1)!(n-x)!}p^x(1-p)^{n-x}\dotso \frac{x}{x!}=\frac{1}{(x-1)!}\\ &=& \displaystyle \sum_{x=0}^{n} \frac{n(n-1)!}{(x-1)!(n-x)!}p^x(1-p)^{n-x}\dotso n!=n(n-1)!\\ &=& \displaystyle \sum_{x=0}^{n} \frac{n(n-1)!}{(x-1)!(n-x)!}pp^{x-1}(1-p)^{n-x}\dotso p^x=pp^{x-1}\\ &=& \displaystyle np\sum_{x=0}^{n} \frac{(n-1)!}{(x-1)!(n-x)!}p^{x-1}(1-p)^{n-x}\\ &=& \displaystyle np\sum_{x=0}^{n} \frac{(n-1)!}{(x-1)!(n-1+1-x)!}p^{x-1}(1-p)^{n-1+1-x}\\ &=& \displaystyle np\sum_{x=0}^{n} \frac{(n-1)!}{(x-1)!(n-1-(x-1))!}p^{x-1}(1-p)^{n-1-(x-1)}\dotso 1-x=-(x-1)\\ &=& \displaystyle np\sum_{x'=0}^{n-1} \frac{(n-1)!}{x'!(n-1-x')!}p^{x'}(1-p)^{n-1-x'}\dotso x'=x-1\\ &=& \displaystyle np\,1 \dotso B(n-1, p)の総和は1に等しい．\\ &=& \displaystyle np\\ &=& \mu \dotso 母集団の期待値\\ \end{array}$$連続型確率変数(continuous random variable) / 期待値(expected value)
$$\begin{array}{rcl}
母集団(population)の確率変数&:&X,Y\\
確率密度凾数(probability \, density \, function, \, PDF)&:&\displaystyle\int_{a}^{b}f_X(x)\mathrm{d}x=P(a \leq x \leq b)\\
&&\displaystyle\int_{-\infty}^{\infty}f_X(x)\mathrm{d}x=1\\
累積分布凾数(cumulative \, distribution \, function, \, CDF)&:&\displaystyle F_X(x)=\int_{-\infty}^{x}f_X(t)\mathrm{d}t\\
&&\displaystyle f_X(x)=\frac{\mathrm{d}}{\mathrm{d}x}F_X(x)\\
同時確率密度凾数(joint\,probability\,density\,function)&:&\displaystyle\int_{a}^{b}\int_{c}^{d}f_{XY}(x, y)\mathrm{d}x\mathrm{d}y=P(a \leq x \leq b,\,c \leq y \leq d)\\
&&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}f_{XY}(x, y)\mathrm{d}x\mathrm{d}y=1\\
周辺確率密度凾数(marginal\, probability\, density\, function)&:&\displaystyle f_X(x)=\int_{-\infty}^{\infty}f_{XY}(x,y)\mathrm{d}y
\end{array}$$
$$\begin{array}{rcl}
E[g(X)]&\equiv&\displaystyle\int_{-\infty}^{\infty} g(x) f_X(x) \mathrm{d}x\\
E[X]&=&\displaystyle\int_{-\infty}^{\infty} (x) f_X(x) \mathrm{d}x\\
E[cX] &=&\displaystyle\int_{-\infty}^{\infty} (c x) f_X(x) \mathrm{d}x\,\dotso\,cは定数\\
&=&c\displaystyle\int_{-\infty}^{\infty} x f_X(x) \mathrm{d}x\\
&=&c E[X] \\
E[X \pm t]&=&\displaystyle\int_{-\infty}^{\infty} (x \pm t) f_X(x) \mathrm{d}x\,\dotso\,tは定数\\
&=&\displaystyle\int_{-\infty}^{\infty} (x f_X(x) \pm t f_X(x)) \mathrm{d}x\\
&=&\displaystyle\int_{-\infty}^{\infty} x f_X(x)
\pm \displaystyle\int_{-\infty}^{\infty} t f_X(x) \mathrm{d}x\\
&=&\displaystyle\int_{-\infty}^{\infty} x f_X(x)
\pm t \displaystyle\int_{-\infty}^{\infty} f_X(x) \mathrm{d}x\\
&=&E[X] \pm t\\
E[X \pm Y]&=&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} (x \pm y) f_{XY}(x, y) \mathrm{d}x\mathrm{d}y\\
&=&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} (x f_{XY}(x, y) \pm y f_{XY}(x, y)) \mathrm{d}x\mathrm{d}y\\
&=&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} x f_{XY}(x, y) \mathrm{d}x\mathrm{d}y
\pm \displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} y f_{XY}(x, y)\mathrm{d}x\mathrm{d}y\\
&=&\displaystyle\int_{-\infty}^{\infty} x \int_{-\infty}^{\infty} f_{XY}(x, y) \mathrm{d}y\,\mathrm{d}x
\pm \displaystyle\int_{-\infty}^{\infty} y \int_{-\infty}^{\infty} f_{XY}(x, y)\mathrm{d}x\,\mathrm{d}y\\
&=&\displaystyle\int_{-\infty}^{\infty} x f_{X}(x) \mathrm{d}x
\pm \displaystyle\int_{-\infty}^{\infty} y f_{Y}(y) \mathrm{d}y\,\dotso\,周辺確率密度凾数を適用\\
&=& E[X] \pm E[Y] \\
\end{array}$$
X,Yが独立の場合
$$\begin{array}{rcl} E[XY]&=&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}(x y)f_{XY}(x, y)\mathrm{d}x\mathrm{d}y\\ &=&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}xyf_{X}(x)f_{Y}(y)\mathrm{d}x\mathrm{d}y \,\dotso\,X,Yが独立\,f_{XY}(x, y)=f_{X}(x)f_{Y}(y)\\ &=&\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}xf_{X}(x)yf_{Y}(y)\mathrm{d}x\mathrm{d}y\\ &=&\displaystyle\left(\int_{-\infty}^{\infty}xf_{X}(x)\mathrm{d}x\right)\left(\int_{-\infty}^{\infty}yf_{Y}(y)\mathrm{d}y\right) \,\dotso\,\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}f(x)g(y)\mathrm{d}x\mathrm{d}y=\left(\int_{-\infty}^{\infty}f(x)\mathrm{d}x\right)\left(\int_{-\infty}^{\infty}g(y)\mathrm{d}y\right)\\ &=&E[X]E[Y] \end{array}$$離散型確率変数(discrete random variable) / 期待値(expected value)
$$\begin{array}{rcl}
母集団(population)の確率変数&:&X,Y\\
確率質量凾数(probability\,mass\,function,\,PMF)&:&f_X(x)=P(X=x_i)\\
&&\displaystyle\sum_{i=1}^{\infty} f_X(x_i)=1\\
累積分布凾数(cumulative\,distribution\,function,\,CDF)&:&\displaystyle F_X(x)=\sum_{i:x_i \leq x} f_X(x_i)\\
同時確率質量凾数(joint\,probability\,mass\,function)&:&f_X(x, y)=P(X=x_i,\,Y=y_j)\\
&&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty} f_{XY}(x_i, y_j)=1\\
周辺確率質量凾数(marginal\, probability\, mass\, function)&:&\displaystyle f_X(x)=\sum_{j=1}^{\infty} f_{XY}(x, y_i)\\
\end{array}$$
$$\begin{array}{rcl}
E[g(X)]&\equiv&\displaystyle\sum_{i=1}^{\infty} g(x_i) f_X(x_i) \\
E[X]&=&\displaystyle\sum_{i=1}^{\infty} (x_i) f_X(x_i) \\
E[cX] &=&\displaystyle\sum_{i=1}^{\infty} (c x_i) f_X(x_i) \,\dotso\,cは定数\\
&=&c\displaystyle\sum_{i=1}^{\infty} x_i f_X(x_i) \\
&=&c E[X] \\
E[X \pm t]&=&\displaystyle\sum_{i=1}^{\infty} (x_i \pm t) f_X(x_i) \,\dotso\,tは定数\\
&=&\displaystyle\sum_{i=1}^{\infty} (x_i f_X(x_i) \pm t f_X(x_i)) \\
&=&\displaystyle\sum_{i=1}^{\infty} x_i f_X(x_i)
\pm \displaystyle\sum_{i=1}^{\infty} t f_X(x_i) \\
&=&\displaystyle\sum_{i=1}^{\infty} x_i f_X(x_i)
\pm t \displaystyle\sum_{i=1}^{\infty} f_X(x_i) \\
&=&E[X] \pm t\\
E[X \pm Y]&=&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty} (x_i \pm y_j) f_{XY}(x_i, y_j) \\
&=&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty} (x_i f_{XY}(x_i, y_j) \pm y_j f_{XY}(x_i, y_j)) \\
&=&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty} x_i f_{XY}(x_i, y_j)
\pm \displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty} y_j f_{XY}(x_i, y_j)\\
&=&\displaystyle\sum_{i=1}^{\infty} x_i \sum_{j=1}^{\infty} f_{XY}(x_i, y_j)
\pm \displaystyle\sum_{j=1}^{\infty} y_j \sum_{i=1}^{\infty}f_{XY}(x_i, y_j)\\
&=&\displaystyle\sum_{i=1}^{\infty} x_i f_{X}(x_i)
\pm \displaystyle\sum_{j=1}^{\infty} y_j f_{Y}(y_j)\,\dotso\,周辺確率質量凾数を適用\\
&=& E[X] \pm E[Y] \\
\end{array}$$
X,Yが独立の場合
$$\begin{array}{rcl} E[XY]&=&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty}(x_i y_j)f_{XY}(x_i, y_j)\\ &=&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty}x_iy_jf_{X}(x_i)f_{Y}(y_i) \,\dotso\,X,Yが独立\,f_{XY}(x, y)=f_{X}(x)f_{Y}(y)\\ &=&\displaystyle\sum_{i=1}^{\infty}\sum_{j=1}^{\infty}x_if_{X}(x_i)y_jf_{Y}(y_i)\\ &=&\displaystyle\left(\sum_{i=1}^{\infty}x_if_{X}(x_i)\right)\left(\sum_{j=1}^{\infty} y_jf_{Y}(y_i)\right) \,\dotso\,\sum_{i=1}^{\infty}\sum_{j=1}^{\infty}a_ib_j=\sum_{i=1}^{\infty}a_i\sum_{j=1}^{\infty}b_j\\ &=&E[X]E[Y] \end{array}$$
登録:
投稿 (Atom)