正規分布に従う互いに独立な標本における分散の最尤推定量を求める
\(x_1,\cdots,x_n\)を\(N\left(\mu,\sigma^2\right)\)からの独立な観測値とする時の\(\sigma^2\)の最尤推定を考える．
確率密度凾数
$$
\begin{eqnarray}
X_k&\sim&N\left(\mu,\sigma^2\right)
\\f\left(x;\mu,\sigma^2\right)&=&\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}
\;\cdots\;正規分布の確率密度凾数
\\f\left(x_1,\cdots,x_n;\mu,\sigma^2\right)&=&\left\{\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_1-\mu)^2}{2\sigma^2}}\right\}
\left\{\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_2-\mu)^2}{2\sigma^2}}\right\}
\cdots
\left\{\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_n-\mu)^2}{2\sigma^2}}\right\}
\\&=&\left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^ne^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2}
\end{eqnarray}
$$
\(\mu\)が既知の場合の\(\sigma^2\)の最尤推定
確率密度凾数から尤度凾数(確率密度凾数に対して，確率変数や既知のパラメータを定数として，未知のパラメータを変数とみなす)の対数をとり，
対数尤度凾数を用意する．
$$
\begin{eqnarray}
\\l\left(\sigma^2;x_1,\cdots,x_n,\mu\right)
&=&\log{ \left\{ \left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^ne^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \right\}}
\\&=&\log{ \left\{ \left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^n\right\}}+\log{ \left\{ e^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \right\}}
\;\cdots\;\log{\left(AB\right)}=\log{\left(A\right)}+\log{\left(B\right)}
\\&=&\log{ \left\{ \left(2\pi\sigma^2\right)^{-\frac{n}{2}}\right\}}+\log{ \left\{ e^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \right\}}
\;\cdots\;\frac{1}{A}=A^{-1}
\\&=&-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\log{ \left( e \right)}
\;\cdots\;\log{\left(A^B\right)}=B\log{\left(A\right)}
\\&=&-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2
\;\cdots\;\log{ \left( e \right)}=1
\end{eqnarray}
$$
極値を考えるために変数である\(\sigma^2\)で微分する(
スコア凾数)．
$$
\begin{eqnarray}
\\\frac{\mathrm{d}}{\mathrm{d} \sigma^2}l\left(\sigma^2;x_1,\cdots,x_n,\mu\right)
&=&\frac{\mathrm{d}}{\mathrm{d} \sigma^2}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\}
\\&=&-\frac{n}{2}\frac{\mathrm{d}}{\mathrm{d} \sigma^2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\frac{\mathrm{d}}{\mathrm{d} \sigma^2}\frac{1}{\sigma^2}
\\&&\;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}\left\{f(x)+g(x)\right\}=\frac{\mathrm{d}}{\mathrm{d}x}f(x)+\frac{\mathrm{d}}{\mathrm{d}x}g(x)
,\;\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x)
\\&=&-\frac{n}{2}\frac{\mathrm{d}}{\mathrm{d} u}\log{ \left( u \right)}\frac{\mathrm{d}u}{\mathrm{d}\sigma^2}-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\frac{\mathrm{d}}{\mathrm{d}v}v^{-1}\frac{\mathrm{d}v}{\mathrm{d}\sigma^2}
\\&&\;\cdots\;u=2\pi\sigma^2,\frac{\mathrm{d}u}{\mathrm{d}\sigma^2}=2\pi, v=\sigma^2,\frac{\mathrm{d}v}{\mathrm{d}\sigma^2}=1
\\&=&-\frac{n}{2}\frac{1}{u}2\pi-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\left(-v^{-2}\right)\cdot1
\\&=&-\frac{n}{2}\frac{1}{2\pi\sigma^2}2\pi-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\left\{-\left(\sigma^2\right)^{-2}\right\}
\;\cdots\;u=2\pi\sigma^2,\;v=\sigma^2
\\&=&-\frac{n}{2}\frac{1}{\sigma^2}-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\left\{\frac{-1}{\left(\sigma^2\right)^2}\right\}
\\&=&-\frac{n}{2\sigma^2}+\frac{1}{2\sigma^4}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}
\;\cdots\;\left(A^B\right)^C=A^{BC}
\end{eqnarray}
$$
この式が0となる\(\sigma^2(=\hat{\sigma}^2)\)を求める(極値である)．
$$
\begin{eqnarray}
\frac{\mathrm{d}}{\mathrm{d} \sigma^2}l\left(\sigma^2;x_1,\cdots,x_n,\mu\right)=-\frac{n}{2\hat{\sigma}^2}+\frac{1}{2\sigma^4}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}&=&0
\\\frac{1}{2\hat{\sigma}^2}\left[-n+\frac{1}{\hat{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\right]&=&0
\\-n+\frac{1}{\hat{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}&=&0
\;\cdots\;\frac{1}{2\hat{\sigma}^2}は\hat{\sigma}^2が有限なら0にならないので0になるのは\left[\right]の中が0の時
\\\frac{1}{\hat{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}&=&n
\\\frac{1}{\hat{\sigma}^2}&=&\frac{n}{\sum_{k=1}^n(x_k-\mu)^2}
\\\hat{\sigma}^2&=&\frac{1}{n}\sum_{k=1}^n(x_k-\mu)^2
\;\cdots\;両辺とも逆数をとった．
\end{eqnarray}
$$
\(\mu\)が未知の場合の\(\sigma^2\)の最尤推定
対数尤度凾数を用意する(上と同じ)．
$$
\begin{eqnarray}
\\l\left(\mu,\sigma^2;x_1,\cdots,x_n\right)
&=&-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2
\end{eqnarray}
$$
極値を考えるために変数である\(\mu,\sigma^2\)でそれぞれ偏微分する(
スコア凾数)．
$$
\begin{eqnarray}
\\\frac{\partial}{\partial \mu}l\left(\mu, \sigma^2;x_1,\cdots,x_n\right)
&=&\frac{\partial}{\partial \mu}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\}
\\&=&\frac{\partial}{\partial \mu}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}\right\}+\frac{\partial}{\partial \mu}\left\{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\}
\\&&\;\cdots\;\frac{\partial}{\partial x}\left\{f(x,y)+g(x,y)\right\} =\frac{\partial}{\partial x}f(x,y) +\frac{\partial}{\partial x}g(x,y)
\\&=&-\frac{1}{2\sigma^2}\frac{\mathrm{d}}{\mathrm{d} \mu}\sum_{k=1}^n(x_k-\mu)^2
\;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}c=0,\;\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x)
\\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n\frac{\mathrm{d}}{\mathrm{d} \mu}(x_k-\mu)^2
\\&&\;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}\sum_{k=1}^nf(x_k)=\frac{\mathrm{d}}{\mathrm{d}x}\left\{f(x_1)+\cdots+f(x_n)\right\}=\frac{\mathrm{d}}{\mathrm{d}x}f(x_1)+\cdots+\frac{\mathrm{d}}{\mathrm{d}x}f(x_n)=\sum_{k=1}^n\frac{\mathrm{d}}{\mathrm{d}x}f(x_k)
\\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n\frac{\mathrm{d}}{\mathrm{d} u}u^2\frac{\mathrm{d}u}{\mathrm{d}\mu}
\;\cdots\;u=x_k-\mu,\frac{\mathrm{d}u}{\mathrm{d}\mu}=-1
\\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n2u\cdot-1
\\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n-2\left(x_k-\mu\right)
\;\cdots\;u=x_k-\mu
\\&=&-\frac{1}{2\sigma^2}(-2)\sum_{k=1}^n\left(x_k-\mu\right)
\\&=&\frac{1}{\sigma^2}\sum_{k=1}^n\left(x_k-\mu\right)
\\\frac{\partial}{\partial \sigma^2}l\left(\mu, \sigma^2;x_1,\cdots,x_n\right)
&=&\frac{\partial}{\partial \sigma^2}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\}
\\&=&-\frac{n}{2\sigma^2}+\frac{1}{2\sigma^4}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}
\;\cdots\;l(\sigma^2;x_1,\cdots,x_n,\mu)の時と同様
\end{eqnarray}
$$
これらの式が0となる連立方程式として\(\mu(=\hat{\mu}),\sigma^2(=\tilde{\sigma}^2)\)を求める(極値である)．
$$
\begin{eqnarray}
\left\{
\begin{array}
\;\frac{1}{\tilde{\sigma}^2} \sum_{k=1}^n\left(x_k-\hat{\mu}\right)&=&0
\\-\frac{n}{2\tilde{\sigma}^2}+\frac{1}{2\tilde{\sigma}^4}\left\{\sum_{k=1}^n(x_k-\hat{\mu})^2\right\}&=&0
\end{array}
\right.
\end{eqnarray}
$$
第一式より
$$
\begin{eqnarray}
\frac{1}{\tilde{\sigma}^2} \sum_{k=1}^n\left(x_k-\hat{\mu}\right)&=&0
\\\sum_{k=1}^n\left(x_k-\hat{\mu}\right)&=&0
\;\cdots\;\frac{1}{\tilde{\sigma}^2}は\tilde{\sigma}^2が有限なら0にならないので\sum以降が0
\\\sum_{k=1}^nx_k-\sum_{k=1}^n\hat{\mu}&=&0
\;\cdots\;\sum (A-B)=\sum A - \sum B
\\\sum_{k=1}^nx_k-n\hat{\mu}&=&0
\;\cdots\;\sum_{k=1}^n c = nc\;(c:定数)
\\-n\hat{\mu}&=&-\sum_{k=1}^nx_k
\\\hat{\mu}&=&\frac{1}{n}\sum_{k=1}^nx_k
\\&=&\bar{x}
\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{標本平均\;\bar{x}=\frac{1}{n}\sum_{k=1}^nx_k}
\end{eqnarray}
$$
第二式の\(\hat{\mu}\)にこれを代入して\(\tilde{\sigma}^2\)の最尤推定量を得る．
$$
\begin{eqnarray}
-\frac{n}{2\tilde{\sigma}^2}+\frac{1}{2\tilde{\sigma}^4}\left\{\sum_{k=1}^n(x_k-\hat{\mu})^2\right\}&=&0
\\-\frac{n}{2\tilde{\sigma}^2}+\frac{1}{2\tilde{\sigma}^4}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}&=&0
\;\cdots\;\hat{\mu}=\bar{x}
\\\frac{1}{2\tilde{\sigma}^2}\left[-n+\frac{1}{\tilde{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}\right]&=&0
\\-n+\frac{1}{\tilde{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}&=&0
\;\cdots\;\frac{1}{2\tilde{\sigma}^2}は\tilde{\sigma}^2が有限なら0にならないので0になるのは\left[\right]の中が0の時
\\\frac{1}{\tilde{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}&=&n
\\\frac{1}{\tilde{\sigma}^2}&=&\frac{n}{\sum_{k=1}^n(x_k-\bar{x})^2}
\\\tilde{\sigma}^2&=&\frac{1}{n}\sum_{k=1}^n(x_k-\bar{x})^2
\\&=&s^2
\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{標本分散\;s^2=\frac{1}{n}\sum_{k=1}^{n} (X_k - \overline{X})^2}
\\&=&\frac{n-1}{n}\hat{\sigma}^2
\;\cdots\;\href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{不偏分散\;\hat{\sigma}^2=\frac{1}{n-1}\sum_{k=1}^{n} (X_k - \overline{X})^2}
\\&&\;\cdots\;これは\href{https://shikitenkai.blogspot.com/2019/07/blog-post_68.html}{標本分散の期待値}でもある．
\end{eqnarray}
$$
0 件のコメント:
コメントを投稿