単回帰モデルの最尤推定

単回帰モデルの最尤推定

単回帰モデル

$$ \begin{eqnarray} y_i&=&\alpha+\beta x_i+\epsilon_i \;(i=1,\cdots,n) \\&&\epsilon_i \overset{iid}{\sim} N(0,\sigma^2)\;\cdots\;独立同一分布(independent\;and\;identically\;distributed;\;IID,\;i.i.d.,\;iid) \end{eqnarray} $$

対数尤度凾数

対数尤度凾数は以下のようになる． $$ \begin{eqnarray} f(y_1,\cdots,y_n;\alpha,\beta,\sigma^2)&=&\prod_{i=1}^{n}\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{1}{2\sigma^2}\left(y_i-\alpha-\beta x_i\right)^2} \\&=&\left(2\pi\right)^{-\frac{n}{2}} \left(\sigma^2\right)^{-\frac{n}{2}} e^{-\frac{1}{2\sigma^2}\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}} \\l(\alpha,\beta,\sigma^2;y_1,\cdots,y_n)&=&\log{\left\{ \left(2\pi\right)^{-\frac{n}{2}} \left(\sigma^2\right)^{-\frac{n}{2}} e^{-\frac{1}{2\sigma^2}\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}} \right\}} \\&=&\log{\left\{ \left(2\pi\right)^{-\frac{n}{2}} \right\}} +\log{\left\{ \left(\sigma^2\right)^{-\frac{n}{2}} \right\}} +\log{\left\{ e^{-\frac{1}{2\sigma^2}\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}} \right\}} \\&=&-\frac{n}{2}\log{\left(2\pi\right)} -\frac{n}{2}\log{\left(\sigma^2\right)} -\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2} \end{eqnarray} $$

スコア凾数

スコア凾数は以下のようになる． $$ \begin{eqnarray} \frac{\partial l}{\partial \alpha} &=&\frac{\partial l}{\partial \alpha}\left\{-\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\} \\&=&-\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\frac{\partial l}{\partial \alpha}\left(y_i-\alpha-\beta x_i\right)^2} \\&=&-\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)(-1)} \\&=&-\frac{-1}{2\sigma^2} \sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)} \\&=&\frac{1}{2\sigma^2} \left(\sum_{i=1}^{n}y_i-\alpha\sum_{i=1}^{n}1-\beta\sum_{i=1}^{n} x_i\right) \\&=&\frac{1}{2\sigma^2} \left(n\bar{y}-n\alpha-n\beta\bar{x}\right) \;\cdots\;\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i,\;\bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_i \\&=&\frac{n}{2\sigma^2} \left(\bar{y}-\alpha-\beta\bar{x}\right) \end{eqnarray} $$ $$ \begin{eqnarray} \frac{\partial l}{\partial \beta} &=&\frac{\partial l}{\partial \beta}\left\{-\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\} \\&=&-\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\frac{\partial l}{\partial \beta}\left(y_i-\alpha-\beta x_i\right)^2} \\&=&-\frac{1}{2\sigma^2} \sum_{i=1}^{n}{2\left(y_i-\alpha-\beta x_i\right)(-x_i)} \\&=&-\frac{-2}{2\sigma^2} \sum_{i=1}^{n}{\left(x_iy_i-x_i\alpha-\beta x_i^2\right)} \\&=&\frac{1}{\sigma^2} \left(\sum_{i=1}^{n}x_iy_i-\sum_{i=1}^{n}x_i\alpha-\sum_{i=1}^{n}\beta x_i^2\right) \\&=&\frac{1}{\sigma^2} \left(\sum_{i=1}^{n}x_iy_i-n\bar{x}\alpha-\beta \sum_{i=1}^{n}x_i^2\right) \;\cdots\;\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i \end{eqnarray} $$ $$ \begin{eqnarray} \frac{\partial l}{\partial \sigma^2} &=&\frac{\partial l}{\partial \sigma^2}\left\{ -\frac{n}{2}\log{\left(\sigma^2\right)} -\frac{1}{2\sigma^2} \sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2} \right\} \\&=& -\frac{n}{2}\frac{\partial l}{\partial \sigma^2}\log{\left(\sigma^2\right)} -\frac{1}{2}\left\{\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\}\frac{\partial l}{\partial \sigma^2}\frac{1}{\sigma^2} \\&=& -\frac{n}{2}\frac{\partial l}{\partial \sigma^2}\log{\left(\sigma^2\right)} -\frac{1}{2}\left\{\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\}\frac{\partial l}{\partial u}\frac{1}{u} \;\cdots\;u=\sigma^2 \\&=& -\frac{n}{2}\frac{\partial l}{\partial \sigma^2}\log{\left(\sigma^2\right)} -\frac{1}{2}\left\{\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\}\left(\frac{-1}{u^2}\right) \\&=& -\frac{n}{2}\frac{1}{\sigma^2} -\frac{1}{2}\left\{\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\}\left(\frac{-1}{\sigma^4}\right) \;\cdots\;u=\sigma^2 \\&=& -\frac{1}{2\sigma^2}\left\{ n-\frac{1}{\sigma^2}\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2} \right\} \end{eqnarray} $$

スコア凾数を連立させる

$$ \begin{eqnarray} \left\{\begin{array}{rcl} \;0&=&\frac{\partial l}{\partial \alpha} \\0&=&\frac{\partial l}{\partial \beta} \\0&=&\frac{\partial l}{\partial \sigma^2} \end{array}\right. \end{eqnarray} $$ $$ \begin{eqnarray} \left\{\begin{array}{rcl} \;0&=&\frac{n}{2\sigma^2} \left(\bar{y}-\alpha-\beta\bar{x}\right) \\0&=&\frac{1}{\sigma^2} \left(\sum_{i=1}^{n}x_iy_i-n\bar{x}\alpha-\beta \sum_{i=1}^{n}x_i^2\right) \\0&=&-\frac{1}{2\sigma^2}\left\{n-\frac{1}{\sigma^2}\sum_{i=1}^{n}{\left(y_i-\alpha-\beta x_i\right)^2}\right\} \end{array}\right. \end{eqnarray} $$ \(\alpha,\beta,\sigma^2\)の推定量を\(\hat{\alpha},\hat{\beta},\hat{\sigma}^2\)とし，\(\hat{\alpha},\hat{\beta},\hat{\sigma}^2\)の最尤推定量(maximum likelihood estimator)を\(\hat{\alpha}_{ML},\hat{\beta}_{ML},\hat{\sigma}^2_{ML}\)とする． $$ \begin{eqnarray} \left\{\begin{array}{rcl} \;0&=&\bar{y}-\hat{\alpha}_{ML}-\hat{\beta}_{ML}\bar{x} \\0&=&\sum_{i=1}^{n}x_iy_i-n\bar{x}\hat{\alpha}_{ML}-\hat{\beta}_{ML} \sum_{i=1}^{n}x_i^2 \\0&=&n-\frac{1}{\sigma^2_{ML}}\sum_{i=1}^{n}{\left(y_i-\hat{\alpha}_{ML}-\hat{\beta}_{ML} x_i\right)^2} \end{array}\right. \end{eqnarray} $$

\(\hat{\alpha}_{ML}\)を求める

$$ \begin{eqnarray} 0&=&\bar{y}-\hat{\alpha}_{ML}-\hat{\beta}_{ML}\bar{x} \\\hat{\alpha}_{ML}&=&\bar{y}-\hat{\beta}_{ML}\bar{x} \end{eqnarray} $$

\(\hat{\beta}_{ML}\)を求める

$$ \begin{eqnarray} 0&=&\sum_{i=1}^{n}x_iy_i-n\bar{x}\hat{\alpha}_{ML}-\hat{\beta}_{ML} \sum_{i=1}^{n}x_i^2 \\&=&\sum_{i=1}^{n}x_iy_i-n\bar{x}\left(\bar{y}-\hat{\beta}_{ML}\bar{x}\right)-\hat{\beta}_{ML} \sum_{i=1}^{n}x_i^2 \;\cdots\;\hat{\alpha}_{ML}=\bar{y}-\hat{\beta}_{ML}\bar{x} \\&=&\sum_{i=1}^{n}x_iy_i-n\bar{x}\bar{y}+n\hat{\beta}_{ML}\bar{x}^2-\hat{\beta}_{ML} \sum_{i=1}^{n}x_i^2 \\&=&\sum_{i=1}^{n}x_iy_i-n\bar{x}\bar{y} -\hat{\beta}_{ML} \left\{ \left( \sum_{i=1}^{n}x_i^2 \right) - n\bar{x}^2\right\} \\&=&\sum_{i=1}^{n}\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) -\hat{\beta}_{ML} \sum_{i=1}^{n}\left(x_i-\bar{x}\right)^2 \\&=&S_{xy} -\hat{\beta}_{ML}\;S_{xx} \;\cdots\;S_{xy}=\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y}),\;S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2 \\\hat{\beta}_{ML}&=&\frac{S_{xy}}{S_{xx}} \\&=&\hat{\beta}\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{正規方程式の解と同じ} \\\hat{\alpha}_{ML}&=&\bar{y}-\hat{\beta}_{ML}\bar{x} \\&=&\bar{y}-\hat{\beta}\bar{x} \\&=&\hat{\alpha}\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/03/blog-post.html}{正規方程式の解と同じ} \end{eqnarray} $$

\(\hat{\sigma}^2_{ML}\)を求める

$$ \begin{eqnarray} 0&=&n-\frac{1}{\sigma^2_{ML}}\sum_{i=1}^{n}{\left(y_i-\hat{\alpha}_{ML}-\hat{\beta}_{ML} x_i\right)^2} \\-n&=&-\frac{1}{\sigma^2_{ML}}\sum_{i=1}^{n}{\left(y_i-\hat{\alpha}_{ML}-\hat{\beta}_{ML} x_i\right)^2} \\-n\sigma^2_{ML}&=&-\sum_{i=1}^{n}{\left(y_i-\hat{\alpha}_{ML}-\hat{\beta}_{ML} x_i\right)^2} \\\hat{\sigma}^2_{ML} &=&\frac{1}{n}\sum_{i=1}^{n}{\left(y_i-\hat{\alpha}_{ML}-\hat{\beta}_{ML} x_i\right)^2} \\&=&\frac{1}{n}\sum_{i=1}^{n}{\left(y_i-\hat{\alpha}-\hat{\beta} x_i\right)^2} \\&=&\frac{1}{n}\sum_{i=1}^{n}{\left(y_i-\hat{y}_i\right)^2} \;\cdots\;\hat{y}_i=\hat{\alpha}+\hat{\beta} x_i \\&=&\frac{1}{n}\sum_{i=1}^{n}{e_i^2} \\&=&\frac{1}{n}(n-2)s^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/09/blog-post.html}{\sum_{i=1}^{n}{e_i^2}=(n-2)s^2,\;s^2=\frac{1}{n-2}\sum_{i=1}^{n}{e_i^2}} \\&=&\frac{n-2}{n}s^2 \end{eqnarray} $$

\(\hat{\alpha}_{ML},\hat{\beta}_{ML},\hat{\sigma}^2_{ML}\)

以上より最尤推定量\(\hat{\alpha}_{ML},\hat{\beta}_{ML},\hat{\sigma}^2_{ML}\)は以下のようになる． $$ \begin{eqnarray} \hat{\beta}_{ML}&=&\frac{S_{xy}}{S_{xx}}=\hat{\beta} \\\hat{\alpha}_{ML}&=&\bar{y}-\hat{\beta}_{ML}\bar{x}=\bar{y}-\hat{\beta}\bar{x}=\hat{\alpha} \\\hat{\sigma}_{ML}^2&=&\frac{n-2}{n}s^2 \end{eqnarray} $$

正規分布に従う互いに独立な標本における分散の最尤推定量を求める

正規分布に従う互いに独立な標本における分散の最尤推定量を求める

\(x_1,\cdots,x_n\)を\(N\left(\mu,\sigma^2\right)\)からの独立な観測値とする時の\(\sigma^2\)の最尤推定を考える．

確率密度凾数

$$ \begin{eqnarray} X_k&\sim&N\left(\mu,\sigma^2\right) \\f\left(x;\mu,\sigma^2\right)&=&\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \;\cdots\;正規分布の確率密度凾数 \\f\left(x_1,\cdots,x_n;\mu,\sigma^2\right)&=&\left\{\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_1-\mu)^2}{2\sigma^2}}\right\} \left\{\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_2-\mu)^2}{2\sigma^2}}\right\} \cdots \left\{\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x_n-\mu)^2}{2\sigma^2}}\right\} \\&=&\left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^ne^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \end{eqnarray} $$

---

\(\mu\)が既知の場合の\(\sigma^2\)の最尤推定

確率密度凾数から尤度凾数(確率密度凾数に対して，確率変数や既知のパラメータを定数として，未知のパラメータを変数とみなす)の対数をとり，対数尤度凾数を用意する． $$ \begin{eqnarray} \\l\left(\sigma^2;x_1,\cdots,x_n,\mu\right) &=&\log{ \left\{ \left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^ne^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \right\}} \\&=&\log{ \left\{ \left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^n\right\}}+\log{ \left\{ e^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \right\}} \;\cdots\;\log{\left(AB\right)}=\log{\left(A\right)}+\log{\left(B\right)} \\&=&\log{ \left\{ \left(2\pi\sigma^2\right)^{-\frac{n}{2}}\right\}}+\log{ \left\{ e^{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2} \right\}} \;\cdots\;\frac{1}{A}=A^{-1} \\&=&-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\log{ \left( e \right)} \;\cdots\;\log{\left(A^B\right)}=B\log{\left(A\right)} \\&=&-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2 \;\cdots\;\log{ \left( e \right)}=1 \end{eqnarray} $$ 極値を考えるために変数である\(\sigma^2\)で微分する(スコア凾数)． $$ \begin{eqnarray} \\\frac{\mathrm{d}}{\mathrm{d} \sigma^2}l\left(\sigma^2;x_1,\cdots,x_n,\mu\right) &=&\frac{\mathrm{d}}{\mathrm{d} \sigma^2}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\} \\&=&-\frac{n}{2}\frac{\mathrm{d}}{\mathrm{d} \sigma^2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\frac{\mathrm{d}}{\mathrm{d} \sigma^2}\frac{1}{\sigma^2} \\&&\;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}\left\{f(x)+g(x)\right\}=\frac{\mathrm{d}}{\mathrm{d}x}f(x)+\frac{\mathrm{d}}{\mathrm{d}x}g(x) ,\;\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x) \\&=&-\frac{n}{2}\frac{\mathrm{d}}{\mathrm{d} u}\log{ \left( u \right)}\frac{\mathrm{d}u}{\mathrm{d}\sigma^2}-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\frac{\mathrm{d}}{\mathrm{d}v}v^{-1}\frac{\mathrm{d}v}{\mathrm{d}\sigma^2} \\&&\;\cdots\;u=2\pi\sigma^2,\frac{\mathrm{d}u}{\mathrm{d}\sigma^2}=2\pi, v=\sigma^2,\frac{\mathrm{d}v}{\mathrm{d}\sigma^2}=1 \\&=&-\frac{n}{2}\frac{1}{u}2\pi-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\left(-v^{-2}\right)\cdot1 \\&=&-\frac{n}{2}\frac{1}{2\pi\sigma^2}2\pi-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\left\{-\left(\sigma^2\right)^{-2}\right\} \;\cdots\;u=2\pi\sigma^2,\;v=\sigma^2 \\&=&-\frac{n}{2}\frac{1}{\sigma^2}-\frac{1}{2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\left\{\frac{-1}{\left(\sigma^2\right)^2}\right\} \\&=&-\frac{n}{2\sigma^2}+\frac{1}{2\sigma^4}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\} \;\cdots\;\left(A^B\right)^C=A^{BC} \end{eqnarray} $$ この式が0となる\(\sigma^2(=\hat{\sigma}^2)\)を求める(極値である)． $$ \begin{eqnarray} \frac{\mathrm{d}}{\mathrm{d} \sigma^2}l\left(\sigma^2;x_1,\cdots,x_n,\mu\right)=-\frac{n}{2\hat{\sigma}^2}+\frac{1}{2\sigma^4}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}&=&0 \\\frac{1}{2\hat{\sigma}^2}\left[-n+\frac{1}{\hat{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}\right]&=&0 \\-n+\frac{1}{\hat{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}&=&0 \;\cdots\;\frac{1}{2\hat{\sigma}^2}は\hat{\sigma}^2が有限なら0にならないので0になるのは\left[\right]の中が0の時 \\\frac{1}{\hat{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\}&=&n \\\frac{1}{\hat{\sigma}^2}&=&\frac{n}{\sum_{k=1}^n(x_k-\mu)^2} \\\hat{\sigma}^2&=&\frac{1}{n}\sum_{k=1}^n(x_k-\mu)^2 \;\cdots\;両辺とも逆数をとった． \end{eqnarray} $$

\(\mu\)が未知の場合の\(\sigma^2\)の最尤推定

対数尤度凾数を用意する(上と同じ)． $$ \begin{eqnarray} \\l\left(\mu,\sigma^2;x_1,\cdots,x_n\right) &=&-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2 \end{eqnarray} $$ 極値を考えるために変数である\(\mu,\sigma^2\)でそれぞれ偏微分する(スコア凾数)． $$ \begin{eqnarray} \\\frac{\partial}{\partial \mu}l\left(\mu, \sigma^2;x_1,\cdots,x_n\right) &=&\frac{\partial}{\partial \mu}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\} \\&=&\frac{\partial}{\partial \mu}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}\right\}+\frac{\partial}{\partial \mu}\left\{-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\} \\&&\;\cdots\;\frac{\partial}{\partial x}\left\{f(x,y)+g(x,y)\right\} =\frac{\partial}{\partial x}f(x,y) +\frac{\partial}{\partial x}g(x,y) \\&=&-\frac{1}{2\sigma^2}\frac{\mathrm{d}}{\mathrm{d} \mu}\sum_{k=1}^n(x_k-\mu)^2 \;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}c=0,\;\frac{\mathrm{d}}{\mathrm{d}x}cf(x)=c\frac{\mathrm{d}}{\mathrm{d}x}f(x) \\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n\frac{\mathrm{d}}{\mathrm{d} \mu}(x_k-\mu)^2 \\&&\;\cdots\;\frac{\mathrm{d}}{\mathrm{d}x}\sum_{k=1}^nf(x_k)=\frac{\mathrm{d}}{\mathrm{d}x}\left\{f(x_1)+\cdots+f(x_n)\right\}=\frac{\mathrm{d}}{\mathrm{d}x}f(x_1)+\cdots+\frac{\mathrm{d}}{\mathrm{d}x}f(x_n)=\sum_{k=1}^n\frac{\mathrm{d}}{\mathrm{d}x}f(x_k) \\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n\frac{\mathrm{d}}{\mathrm{d} u}u^2\frac{\mathrm{d}u}{\mathrm{d}\mu} \;\cdots\;u=x_k-\mu,\frac{\mathrm{d}u}{\mathrm{d}\mu}=-1 \\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n2u\cdot-1 \\&=&-\frac{1}{2\sigma^2}\sum_{k=1}^n-2\left(x_k-\mu\right) \;\cdots\;u=x_k-\mu \\&=&-\frac{1}{2\sigma^2}(-2)\sum_{k=1}^n\left(x_k-\mu\right) \\&=&\frac{1}{\sigma^2}\sum_{k=1}^n\left(x_k-\mu\right) \\\frac{\partial}{\partial \sigma^2}l\left(\mu, \sigma^2;x_1,\cdots,x_n\right) &=&\frac{\partial}{\partial \sigma^2}\left\{-\frac{n}{2}\log{ \left( 2\pi\sigma^2 \right)}-\frac{1}{2\sigma^2}\sum_{k=1}^n(x_k-\mu)^2\right\} \\&=&-\frac{n}{2\sigma^2}+\frac{1}{2\sigma^4}\left\{\sum_{k=1}^n(x_k-\mu)^2\right\} \;\cdots\;l(\sigma^2;x_1,\cdots,x_n,\mu)の時と同様 \end{eqnarray} $$ これらの式が0となる連立方程式として\(\mu(=\hat{\mu}),\sigma^2(=\tilde{\sigma}^2)\)を求める(極値である)． $$ \begin{eqnarray} \left\{ \begin{array} \;\frac{1}{\tilde{\sigma}^2} \sum_{k=1}^n\left(x_k-\hat{\mu}\right)&=&0 \\-\frac{n}{2\tilde{\sigma}^2}+\frac{1}{2\tilde{\sigma}^4}\left\{\sum_{k=1}^n(x_k-\hat{\mu})^2\right\}&=&0 \end{array} \right. \end{eqnarray} $$ 第一式より $$ \begin{eqnarray} \frac{1}{\tilde{\sigma}^2} \sum_{k=1}^n\left(x_k-\hat{\mu}\right)&=&0 \\\sum_{k=1}^n\left(x_k-\hat{\mu}\right)&=&0 \;\cdots\;\frac{1}{\tilde{\sigma}^2}は\tilde{\sigma}^2が有限なら0にならないので\sum以降が0 \\\sum_{k=1}^nx_k-\sum_{k=1}^n\hat{\mu}&=&0 \;\cdots\;\sum (A-B)=\sum A - \sum B \\\sum_{k=1}^nx_k-n\hat{\mu}&=&0 \;\cdots\;\sum_{k=1}^n c = nc\;(c:定数) \\-n\hat{\mu}&=&-\sum_{k=1}^nx_k \\\hat{\mu}&=&\frac{1}{n}\sum_{k=1}^nx_k \\&=&\bar{x} \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{標本平均\;\bar{x}=\frac{1}{n}\sum_{k=1}^nx_k} \end{eqnarray} $$ 第二式の\(\hat{\mu}\)にこれを代入して\(\tilde{\sigma}^2\)の最尤推定量を得る． $$ \begin{eqnarray} -\frac{n}{2\tilde{\sigma}^2}+\frac{1}{2\tilde{\sigma}^4}\left\{\sum_{k=1}^n(x_k-\hat{\mu})^2\right\}&=&0 \\-\frac{n}{2\tilde{\sigma}^2}+\frac{1}{2\tilde{\sigma}^4}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}&=&0 \;\cdots\;\hat{\mu}=\bar{x} \\\frac{1}{2\tilde{\sigma}^2}\left[-n+\frac{1}{\tilde{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}\right]&=&0 \\-n+\frac{1}{\tilde{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}&=&0 \;\cdots\;\frac{1}{2\tilde{\sigma}^2}は\tilde{\sigma}^2が有限なら0にならないので0になるのは\left[\right]の中が0の時 \\\frac{1}{\tilde{\sigma}^2}\left\{\sum_{k=1}^n(x_k-\bar{x})^2\right\}&=&n \\\frac{1}{\tilde{\sigma}^2}&=&\frac{n}{\sum_{k=1}^n(x_k-\bar{x})^2} \\\tilde{\sigma}^2&=&\frac{1}{n}\sum_{k=1}^n(x_k-\bar{x})^2 \\&=&s^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{標本分散\;s^2=\frac{1}{n}\sum_{k=1}^{n} (X_k - \overline{X})^2} \\&=&\frac{n-1}{n}\hat{\sigma}^2 \;\cdots\;\href{https://shikitenkai.blogspot.com/2019/07/specimen-random-variable.html}{不偏分散\;\hat{\sigma}^2=\frac{1}{n-1}\sum_{k=1}^{n} (X_k - \overline{X})^2} \\&&\;\cdots\;これは\href{https://shikitenkai.blogspot.com/2019/07/blog-post_68.html}{標本分散の期待値}でもある． \end{eqnarray} $$

スコア凾数

スコア凾数

• \(f(x;\theta)\)を\(\theta\)をパラメタとした\(x\)の確率密度分布とする．
• \(f(\theta;x)\)を\(\theta\)の凾数とみた場合を尤度凾数と呼ぶ．
• \(\log{f(\theta;x)}\)を対数尤度凾数と呼ぶ．
• \(\frac{\partial \log{f(\theta;x)}}{\partial \theta}\)をスコア凾数と呼ぶ．
• 対数尤度凾数をパラメタ(\(\theta\))で微分したスコア凾数が\(0\)となるパラメタが，尤度を極値とするパラメタとなる(対数は単調増加凾数)．