尤度凾数の期待値の平均情報量(連続)

尤度凾数の期待値の平均情報量(連続)

$$ \begin{eqnarray} G_n &=&-\mathrm{E}_X\left[\log\left(\; \mathrm{E}_\Theta\left[q(X;\theta)\right] \;\right)\right] \;\cdots\;尤度凾数の期待値の平均情報量\\ &=&-\mathrm{E}_X\left[\log\left(\; \mathrm{E}_\Theta\left[q(X;\theta_0)\exp\left(-f(X_i,\theta_0,\theta)\right)\right] \;\right)\right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/05/blog-post.html}{q(X;\theta)=q(X;\theta_0)\exp\left(-f(X_i,\theta_0,\theta)\right)}\\ &=&-\mathrm{E}_X\left[\log\left(\; q(X;\theta_0)\mathrm{E}_\Theta\left[ \exp\left(-f(X_i,\theta_0,\theta)\right) \right] \;\right)\right] \;\cdots\;\mathrm{E}\left[cX\right]=c\mathrm{E}\left[X\right]\\ &=&-\mathrm{E}_X\left[ \log\left( q(X;\theta_0) \right) +\log\left( \mathrm{E}_\Theta\left[ \exp\left(-f(X_i,\theta_0,\theta)\right) \right] \right) \right] \;\cdots\;\log(AB)=\log(A)+\log(B)\\ &=&-\mathrm{E}_X\left[ \log\left( q(X;\theta_0) \right) \right] -\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \exp\left(-f(X_i,\theta_0,\theta)\right) \right] \right) \right] \;\cdots\;\mathrm{E}\left[X+Y\right]=\mathrm{E}\left[X\right]+\mathrm{E}\left[Y\right]\\ &=&L(\theta_0) -\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \exp\left(-f(X_i,\theta_0,\theta)\right) \right] \right) \right] \;\cdots\;\href{https://shikitenkai.blogspot.com/2020/05/blog-post.html}{-\mathrm{E}_X\left[ \log\left( q(X;\theta_0) \right) \right]=L(\theta_0)}\\ &=&L(\theta_0)+G_n^{(0)} \;\cdots\;G_n^{(0)}=-\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \exp\left(-f(X_i,\theta_0,\theta)\right) \right] \right) \right]:正規化された尤度凾数の期待値の平均情報量\\ \end{eqnarray} $$

---

$$ \begin{eqnarray} G_n^{(0)} &=&-\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \exp\left(-f(X_i,\theta_0,\theta)\right) \right] \right) \right]\\ &=&-\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \exp\left(-\log{\frac{q(X_i;\theta_0)}{q(X_i;\theta)}}\right) \right] \right) \right]\\ &=&-\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \exp\left(\log{\frac{q(X_i;\theta)}{q(X_i;\theta_0)}}\right) \right] \right) \right]\\ &=&-\mathrm{E}_X\left[ \log\left( \mathrm{E}_\Theta\left[ \frac{q(X_i;\theta)}{q(X_i;\theta_0)} \right] \right) \right]\\ &=&-\mathrm{E}_X\left[ \log\left( \frac{\mathrm{E}_\Theta\left[q(X_i;\theta)\right]}{q(X_i;\theta_0)} \right) \right]\\ &=&\mathrm{E}_X\left[ -\log\left( \frac{\mathrm{E}_\Theta\left[q(X_i;\theta)\right]}{q(X_i;\theta_0)} \right) \right]\\ &=&\mathrm{E}_X\left[ \log\left( \frac{q(X_i;\theta_0)}{\mathrm{E}_\Theta\left[q(X_i;\theta)\right]} \right) \right]\\ \end{eqnarray} $$