正規分布に従う互いに独立な標本における分散の最尤推定量を求める

正規分布に従う互いに独立な標本における分散の最尤推定量を求める

$x_1,\cdots ,x_n$を$N(\mu ,{\sigma }^2)$からの独立な観測値とする時の${\sigma }^2$の最尤推定を考える．

確率密度凾数

$$\begin{array}{rcl}{X}_k& \sim & N(\mu ,{\sigma }^2)\\ f(x;\mu ,{\sigma }^2)& =& \frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}\cdots \mathrm{正}\mathrm{規}\mathrm{分}\mathrm{布}\mathrm{の}\mathrm{確}\mathrm{率}\mathrm{密}\mathrm{度}\mathrm{凾}\mathrm{数}\\ f(x_1,\cdots ,x_n;\mu ,{\sigma }^2)& =& \left\{\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x_1-\mu {)}^2}{2{\sigma }^2}}\right\}\left\{\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x_2-\mu {)}^2}{2{\sigma }^2}}\right\}\cdots \left\{\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x_n-\mu {)}^2}{2{\sigma }^2}}\right\}\\ & =& {\left(\frac{1}{\sqrt{2\pi {\sigma }^2}}\right)}^n{e}^{-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2}\end{array}$$

---

$\mu$が既知の場合の${\sigma }^2$の最尤推定

確率密度凾数から尤度凾数(確率密度凾数に対して，確率変数や既知のパラメータを定数として，未知のパラメータを変数とみなす)の対数をとり，対数尤度凾数を用意する． $$\begin{array}{r}\\ l({\sigma }^2;x_1,\cdots ,x_n,\mu )& =& \log \left\{{\left(\frac{1}{\sqrt{2\pi {\sigma }^2}}\right)}^n{e}^{-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2}\right\}\\ & =& \log \left\{{\left(\frac{1}{\sqrt{2\pi {\sigma }^2}}\right)}^n\right\}+\log \left\{e^{-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2}\right\}\cdots \log \left(AB\right)=\log \left(A\right)+\log \left(B\right)\\ & =& \log \left\{{\left(2\pi {\sigma }^2\right)}^{-\frac{n}{2}}\right\}+\log \left\{e^{-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2}\right\}\cdots \frac{1}{A}=A^{-1}\\ & =& -\frac{n}{2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\log \left(e\right)\cdots \log \left(A^B\right)=B\log \left(A\right)\\ & =& -\frac{n}{2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\cdots \log \left(e\right)=1\end{array}$$ 極値を考えるために変数である${\sigma }^2$で微分する(スコア凾数)． $$\begin{array}{r}\\ \frac{\mathrm{d}}{\mathrm{d}{\sigma }^2}l({\sigma }^2;x_1,\cdots ,x_n,\mu )& =& \frac{\mathrm{d}}{\mathrm{d}{\sigma }^2}\{-\frac{n}{2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\}\\ & =& -\frac{n}{2}\frac{\mathrm{d}}{\mathrm{d}{\sigma }^2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}\frac{\mathrm{d}}{\mathrm{d}{\sigma }^2}\frac{1}{{\sigma }^2}\\ & & \cdots \frac{\mathrm{d}}{\mathrm{d}x}\{f(x)+g(x)\}=\frac{\mathrm{d}}{\mathrm{d}x}f(x)+\frac{\mathrm{d}}{\mathrm{d}x}g(x),\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x)\\ & =& -\frac{n}{2}\frac{\mathrm{d}}{\mathrm{d}u}\log \left(u\right)\frac{\mathrm{d}u}{\mathrm{d}{\sigma }^2}-\frac{1}{2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}\frac{\mathrm{d}}{\mathrm{d}v}{v}^{-1}\frac{\mathrm{d}v}{\mathrm{d}{\sigma }^2}\\ & & \cdots u=2\pi {\sigma }^2,\frac{\mathrm{d}u}{\mathrm{d}{\sigma }^2}=2\pi ,v={\sigma }^2,\frac{\mathrm{d}v}{\mathrm{d}{\sigma }^2}=1\\ & =& -\frac{n}{2}\frac{1}{u}2\pi -\frac{1}{2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}(-v^{-2})\cdot 1\\ & =& -\frac{n}{2}\frac{1}{2\pi {\sigma }^2}2\pi -\frac{1}{2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}\{-{\left({\sigma }^2\right)}^{-2}\}\cdots u=2\pi {\sigma }^2,v={\sigma }^2\\ & =& -\frac{n}{2}\frac{1}{{\sigma }^2}-\frac{1}{2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}\left\{\frac{-1}{{\left({\sigma }^2\right)}^2}\right\}\\ & =& -\frac{n}{2{\sigma }^2}+\frac{1}{2{\sigma }^4}\{\sum _{k=1}^n(x_k-\mu {)}^2\}\cdots {\left(A^B\right)}^C=A^{BC}\end{array}$$ この式が0となる${\sigma }^2(={\hat{\sigma }}^2)$を求める(極値である)． $$\begin{array}{rcl}\frac{\mathrm{d}}{\mathrm{d}{\sigma }^2}l({\sigma }^2;x_1,\cdots ,x_n,\mu )=-\frac{n}{2{\hat{\sigma }}^2}+\frac{1}{2{\sigma }^4}\{\sum _{k=1}^n(x_k-\mu {)}^2\}& =& 0\\ \frac{1}{2{\hat{\sigma }}^2}[-n+\frac{1}{{\hat{\sigma }}^2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}]& =& 0\\ -n+\frac{1}{{\hat{\sigma }}^2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}& =& 0\cdots \frac{1}{2{\hat{\sigma }}^2}\mathrm{は}{\hat{\sigma }}^2\mathrm{が}\mathrm{有}\mathrm{限}\mathrm{な}\mathrm{ら}0\mathrm{に}\mathrm{な}\mathrm{ら}\mathrm{な}\mathrm{い}\mathrm{の}\mathrm{で}0\mathrm{に}\mathrm{な}\mathrm{る}\mathrm{の}\mathrm{は}\left[\right]\mathrm{の}\mathrm{中}\mathrm{が}0\mathrm{の}\mathrm{時}\\ \frac{1}{{\hat{\sigma }}^2}\{\sum _{k=1}^n(x_k-\mu {)}^2\}& =& n\\ \frac{1}{{\hat{\sigma }}^2}& =& \frac{n}{\sum _{k=1}^n(x_k-\mu {)}^2}\\ {\hat{\sigma }}^2& =& \frac{1}{n}\sum _{k=1}^n(x_k-\mu {)}^2\cdots \mathrm{両}\mathrm{辺}\mathrm{と}\mathrm{も}\mathrm{逆}\mathrm{数}\mathrm{を}\mathrm{と}\mathrm{っ}\mathrm{た}．\end{array}$$

$\mu$が未知の場合の${\sigma }^2$の最尤推定

対数尤度凾数を用意する(上と同じ)． $$\begin{array}{r}\\ l(\mu ,{\sigma }^2;x_1,\cdots ,x_n)& =& -\frac{n}{2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\end{array}$$ 極値を考えるために変数である$\mu ,{\sigma }^2$でそれぞれ偏微分する(スコア凾数)． $$\begin{array}{r}\\ \frac{\partial }{\partial \mu }l(\mu ,{\sigma }^2;x_1,\cdots ,x_n)& =& \frac{\partial }{\partial \mu }\{-\frac{n}{2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\}\\ & =& \frac{\partial }{\partial \mu }\{-\frac{n}{2}\log \left(2\pi {\sigma }^2\right)\}+\frac{\partial }{\partial \mu }\{-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\}\\ & & \cdots \frac{\partial }{\partial x}\{f(x,y)+g(x,y)\}=\frac{\partial }{\partial x}f(x,y)+\frac{\partial }{\partial x}g(x,y)\\ & =& -\frac{1}{2{\sigma }^2}\frac{\mathrm{d}}{\mathrm{d}\mu }\sum _{k=1}^n(x_k-\mu {)}^2\cdots \frac{\mathrm{d}}{\mathrm{d}x}c=0,\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x)\\ & =& -\frac{1}{2{\sigma }^2}\sum _{k=1}^n\frac{\mathrm{d}}{\mathrm{d}\mu }(x_k-\mu {)}^2\\ & & \cdots \frac{\mathrm{d}}{\mathrm{d}x}\sum _{k=1}^{n}f(x_k)=\frac{\mathrm{d}}{\mathrm{d}x}\{f(x_1)+\cdots +f(x_n)\}=\frac{\mathrm{d}}{\mathrm{d}x}f(x_1)+\cdots +\frac{\mathrm{d}}{\mathrm{d}x}f(x_n)=\sum _{k=1}^n\frac{\mathrm{d}}{\mathrm{d}x}f(x_k)\\ & =& -\frac{1}{2{\sigma }^2}\sum _{k=1}^n\frac{\mathrm{d}}{\mathrm{d}u}{u}^2\frac{\mathrm{d}u}{\mathrm{d}\mu }\cdots u=x_k-\mu ,\frac{\mathrm{d}u}{\mathrm{d}\mu }=-1\\ & =& -\frac{1}{2{\sigma }^2}\sum _{k=1}^{n}2u\cdot -1\\ & =& -\frac{1}{2{\sigma }^2}\sum _{k=1}^n-2(x_k-\mu )\cdots u=x_k-\mu \\ & =& -\frac{1}{2{\sigma }^2}(-2)\sum _{k=1}^n(x_k-\mu )\\ & =& \frac{1}{{\sigma }^2}\sum _{k=1}^n(x_k-\mu )\\ \frac{\partial }{\partial {\sigma }^2}l(\mu ,{\sigma }^2;x_1,\cdots ,x_n)& =& \frac{\partial }{\partial {\sigma }^2}\{-\frac{n}{2}\log \left(2\pi {\sigma }^2\right)-\frac{1}{2{\sigma }^2}\sum _{k=1}^n(x_k-\mu {)}^2\}\\ & =& -\frac{n}{2{\sigma }^2}+\frac{1}{2{\sigma }^4}\{\sum _{k=1}^n(x_k-\mu {)}^2\}\cdots l({\sigma }^2;x_1,\cdots ,x_n,\mu )\mathrm{の}\mathrm{時}\mathrm{と}\mathrm{同}\mathrm{様}\end{array}$$ これらの式が0となる連立方程式として$\mu (=\hat{\mu }),{\sigma }^2(={\tilde{\sigma }}^2)$を求める(極値である)． $$\begin{array}{r}\{\begin{array}{}\frac{1}{{\tilde{\sigma }}^2}\sum _{k=1}^n(x_k-\hat{\mu })& =& 0\\ -\frac{n}{2{\tilde{\sigma }}^2}+\frac{1}{2{\tilde{\sigma }}^4}\{\sum _{k=1}^n(x_k-\hat{\mu }{)}^2\}& =& 0\end{array}\end{array}$$ 第一式より $$\begin{array}{rcl}\frac{1}{{\tilde{\sigma }}^2}\sum _{k=1}^n(x_k-\hat{\mu })& =& 0\\ \sum _{k=1}^n(x_k-\hat{\mu })& =& 0\cdots \frac{1}{{\tilde{\sigma }}^2}\mathrm{は}{\tilde{\sigma }}^2\mathrm{が}\mathrm{有}\mathrm{限}\mathrm{な}\mathrm{ら}0\mathrm{に}\mathrm{な}\mathrm{ら}\mathrm{な}\mathrm{い}\mathrm{の}\mathrm{で}\sum \mathrm{以}\mathrm{降}\mathrm{が}0\\ \sum _{k=1}^n{x}_k-\sum _{k=1}^n\hat{\mu }& =& 0\cdots \sum (A-B)=\sum A-\sum B\\ \sum _{k=1}^n{x}_k-n\hat{\mu }& =& 0\cdots \sum _{k=1}^{n}c=nc(c:\mathrm{定}\mathrm{数})\\ -n\hat{\mu }& =& -\sum _{k=1}^n{x}_k\\ \hat{\mu }& =& \frac{1}{n}\sum _{k=1}^n{x}_k\\ & =& \overline{x}\cdots \mathrm{標}\mathrm{本}\mathrm{平}\mathrm{均}\overline{x}=\frac{1}{n}\sum _{k=1}^n{x}_k\end{array}$$ 第二式の$\hat{\mu }$にこれを代入して${\tilde{\sigma }}^2$の最尤推定量を得る． $$\begin{array}{rcl}-\frac{n}{2{\tilde{\sigma }}^2}+\frac{1}{2{\tilde{\sigma }}^4}\{\sum _{k=1}^n(x_k-\hat{\mu }{)}^2\}& =& 0\\ -\frac{n}{2{\tilde{\sigma }}^2}+\frac{1}{2{\tilde{\sigma }}^4}\{\sum _{k=1}^n(x_k-\overline{x}{)}^2\}& =& 0\cdots \hat{\mu }=\overline{x}\\ \frac{1}{2{\tilde{\sigma }}^2}[-n+\frac{1}{{\tilde{\sigma }}^2}\{\sum _{k=1}^n(x_k-\overline{x}{)}^2\}]& =& 0\\ -n+\frac{1}{{\tilde{\sigma }}^2}\{\sum _{k=1}^n(x_k-\overline{x}{)}^2\}& =& 0\cdots \frac{1}{2{\tilde{\sigma }}^2}\mathrm{は}{\tilde{\sigma }}^2\mathrm{が}\mathrm{有}\mathrm{限}\mathrm{な}\mathrm{ら}0\mathrm{に}\mathrm{な}\mathrm{ら}\mathrm{な}\mathrm{い}\mathrm{の}\mathrm{で}0\mathrm{に}\mathrm{な}\mathrm{る}\mathrm{の}\mathrm{は}\left[\right]\mathrm{の}\mathrm{中}\mathrm{が}0\mathrm{の}\mathrm{時}\\ \frac{1}{{\tilde{\sigma }}^2}\{\sum _{k=1}^n(x_k-\overline{x}{)}^2\}& =& n\\ \frac{1}{{\tilde{\sigma }}^2}& =& \frac{n}{\sum _{k=1}^n(x_k-\overline{x}{)}^2}\\ {\tilde{\sigma }}^2& =& \frac{1}{n}\sum _{k=1}^n(x_k-\overline{x}{)}^2\\ & =& s^2\cdots \mathrm{標}\mathrm{本}\mathrm{分}\mathrm{散}{s}^2=\frac{1}{n}\sum _{k=1}^n(X_k-\bar{X}{)}^2\\ & =& \frac{n-1}{n}{\hat{\sigma }}^2\cdots \mathrm{不}\mathrm{偏}\mathrm{分}\mathrm{散}{\hat{\sigma }}^2=\frac{1}{n-1}\sum _{k=1}^n(X_k-\bar{X}{)}^2\\ & & \cdots \mathrm{こ}\mathrm{れ}\mathrm{は}\mathrm{標}\mathrm{本}\mathrm{分}\mathrm{散}\mathrm{の}\mathrm{期}\mathrm{待}\mathrm{値}\mathrm{で}\mathrm{も}\mathrm{あ}\mathrm{る}．\end{array}$$