クラメール-ラオの下限

クラメール-ラオの下限

$$ \begin{eqnarray} \mathrm{E}\left[\hat{\theta}\right] &=&\theta\;\cdots\;\hat{\theta}を\thetaの不偏推定量とする\\ \frac{\partial}{\partial \theta}\mathrm{E}\left[\hat{\theta}\right] &=&\frac{\partial}{\partial \theta}\theta\;\cdots\;両辺を\thetaで微分する\\ \frac{\partial}{\partial \theta}\int{\hat{\theta}f(x;\theta)\mathrm{d}x} &=&1 \;\cdots\;右辺は\thetaを\thetaで微分したので1\\ &&\;\cdots\;左辺は期待値の定義通りE\left[\hat{\theta}\right]=\int_X \hat{\theta} f(x;\theta) \mathrm{d}xと展開した\\ &&\;\cdots\;f(x;\theta)は\thetaをパラメタとしたxの確率密度分布である\\ \end{eqnarray} $$ $$ \begin{eqnarray} 1=\frac{\partial}{\partial \theta}\int{\hat{\theta}f(x;\theta)\mathrm{d}x} &=&\int{\hat{\theta}\frac{\partial f(x;\theta)}{\partial \theta}\mathrm{d}x} \;\cdots\;微分と積分の順番を入れ替えられる場合を考える\\ &=&\int{\hat{\theta} \frac{\partial f(x;\theta)}{\partial \theta}\left\{\frac{1}{f(x;\theta)}f(x;\theta)\right\}\mathrm{d}x}\\ &=&\int{\hat{\theta} \left\{\frac{\partial f(x;\theta)}{\partial \theta}\frac{1}{f(x;\theta)}\right\}f(x;\theta)\mathrm{d}x}\\ &=&\int{\hat{\theta} \frac{\partial \log{f(x;\theta)}}{\partial \theta} f(x;\theta)\mathrm{d}x} \;\cdots\; \frac{\partial f(x;\theta)}{\partial \theta} \frac{1}{f(x;\theta)}=\frac{\partial \log{f(x;\theta)}}{\partial \theta}\\ &=&\mathrm{E}\left[\hat{\theta} \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]\\ &=&\mathrm{E}\left[\hat{\theta} \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right] -\theta \mathrm{E}\left[\frac{\partial \log{f(x;\theta)}}{\partial \theta} \right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/04/blog-post_19.html}{スコア凾数の期待値は0\left(\mathrm{E}\left[\frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]=0\right)}なので加えても変わらない\\ &=&\mathrm{E}\left[\hat{\theta} \frac{\partial \log{f(x;\theta)}}{\partial \theta} -\theta \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right] \;\cdots\;c\mathrm{E}\left[X\right]=\mathrm{E}\left[cX\right],\;\mathrm{E}\left[X\right]+\mathrm{E}\left[Y\right]=\mathrm{E}\left[X+Y\right]\\ &=&\mathrm{E}\left[(\hat{\theta}-\theta) \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]\\ &=&\mathrm{E}\left[ \left((\hat{\theta}-\theta)-\mathrm{E}\left[(\hat{\theta}-\theta)\right]\right) \left(\frac{\partial \log{f(x;\theta)}}{\partial \theta}-\mathrm{E}\left[\frac{\partial \log{f(x;\theta)}}{\partial \theta}\right]\right) \right] \;\cdots\;\mathrm{E}\left[(\hat{\theta}-\theta)\right]=\mathrm{E}\left[\hat{\theta}\right]-\mathrm{E}\left[\theta\right]=\theta-\theta=0 ,\;\mathrm{E}\left[\frac{\partial \log{f(x;\theta)}}{\partial \theta}\right]=0\\ &=&\mathrm{Cov}\left[\hat{\theta}, \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right] \;\cdots\;E\left[\left(X-E\left[X\right]\right) \left(Y-E\left[Y\right]\right) \right]\\ \end{eqnarray} $$ $$ \begin{eqnarray} -1\leq\rho&\leq&1\;\cdots\;\rhoを相関係数とする\\ -1\leq\frac{\mathrm{Cov}\left[X,\;Y\right]}{ \sqrt{ \mathrm{V}\left[ X \right] } \sqrt{ \mathrm{V}\left[ Y \right] } }&\leq&1\;\cdots\;\rho=\frac{\mathrm{Cov}\left[X,\;Y\right]}{ \sqrt{ \mathrm{V}\left[ X \right] } \sqrt{ \mathrm{V}\left[ Y \right] } }\\ -1\leq\frac{\mathrm{Cov}\left[\hat{\theta}, \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]} {\sqrt{\mathrm{V}\left[\hat{\theta}\right]}\sqrt{\mathrm{V}\left[ \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]}}&\leq&1 \;\cdots\;\rho=\frac{\mathrm{Cov}\left[\hat{\theta}, \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]} {\sqrt{\mathrm{V}\left[\hat{\theta}\right]}\sqrt{\mathrm{V}\left[ \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]}}\\ \end{eqnarray} $$ $$ \begin{eqnarray} \rho^2&\leq&1\\ \frac{\mathrm{Cov}\left[\hat{\theta}, \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]^2} {\mathrm{V}\left[\hat{\theta}\right]\mathrm{V}\left[ \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]}&\leq&1\\ \frac{1^2}{\mathrm{V}\left[\hat{\theta}\right]\mathrm{V}\left[ \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]}&\leq&1 \;\cdots\;\mathrm{Cov}\left[\hat{\theta}, \frac{\partial \log{f(x;\theta)}}{\partial \theta} \right]=1\\ \frac{1}{\mathrm{V}\left[ \frac{ \partial \log{ f(x;\theta) }}{ \partial \theta } \right]} &\leq& \mathrm{V} \left[ \hat{\theta} \right]\;\cdots\;\href{https://shikitenkai.blogspot.com/2020/04/blog-post_76.html}{\mathrm{V} \left[ \hat{\theta} \right]:不偏推定量の分散}\\ \mathrm{V}\left[ \frac{ \partial \log{ f(x;\theta) }}{ \partial \theta } \right]^{-1} &\leq& \mathrm{V} \left[ \hat{\theta} \right]\\ \mathcal{I}^{-1}&\leq& \mathrm{V} \left[ \hat{\theta} \right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/04/blog-post_19.html}{\mathcal{I}=\mathrm{V}\left[\frac{ \partial \log{ f(x;\theta) }}{ \partial \theta }\right]:フィッシャー情報量}\\ \end{eqnarray} $$ パラメタの推定(不偏推定量)の分散(パラメタの推定値と真の値(標本が従っている確率分布凾数のパラメタという意味で)との平均二乗誤差)は，フィッシャー情報量の逆数以下にはならないことを示す．