Let $$X_1\sim N(\mu_{X_1},\sigma_{X_2}^2)$$ $$X_2\sim N(\mu_{X_2}, \sigma_{X_2}^2)$$ where $\mu_{X_2}=c+aX_1$. Also, I have data $D$ (with missing values on $X_1,X_2$).

How can I update/estimate the parameters $\mu_{X_1},\sigma_{X_1},\mu_{X_2},a,c,\sigma_{X_2}$ using EM?i.e. what is the formula for updating $\sigma_{X_2}$?

My model is a conditional Gaussian, which is a conditional form of a bivariate Gaussian $(X_1,X_2)$ with mean vector $(\mu_1,\mu_2)^\top$ and covariance matrix $$\left( \begin{matrix} \Sigma _{11} & \Sigma _{12} \\ \Sigma _{21} & \Sigma _{22} \\ \end{matrix} \right)$$

Here is a reference to convert bivariate Gaussian to conditional Gaussian: $$\mu_{2|1}=\mu_{2}+\Sigma_{21}\Sigma_{11}^{-1}(X_1-\mu_1)\quad,\quad \Sigma_{22|1}=\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}$$ which yields my model.

It seems that when $X_1$ has different observations and $X_2$ is unobserved, the variance of $X_2$ remains unchanged.So how to update the $\sigma_{x_2}$?

Initial setting for the model $$X_1\sim N(5,7)\quad,\quad X_2\sim N(0.5X_1,8)$$



$$\quad X_2:\operatorname{NA\,\,NA\,\,3}$$


When $X_1$ is unobserved, at iteration $k=1$ of EM, the posterior mean value (when $X_2=3$) is $5.18$ by using an inference algorithm, i.e. Junction tree/kalman filter. Then the sufficient statistics for $X_1$ is:

$s_1=\Sigma_{i=1}^nx_{i1}=9+4+5.18$ and $s_{11}=\Sigma_{i=1}^nx_{i1}^2=9^2+4^2+(5.18^2+\sigma_{11.2})$

where $\sigma_{11.2}$ is the posterior conditional variance of $X_1$ given $X_2$, when $X_1$ is unobserved.

$s_2=\Sigma_{i=1}^nx_{i2}$ and $s_{22}=\Sigma_{i=1}^nx_{i2}^2$. When $X_2$ is unobserved a term $\sigma_{22.1}$ is also added to $x_{i2}^2$.

So parameters $\mu_{X_1}$ and $\sigma_{X_1}$ are obtained. Similarly, posterior mean of unobserved $X_2$ is also obtained by using i.e. Junction tree inference.

The sufficient statistics for $s_{12}=\Sigma_{i=1}^nx_{i1}x_{i2}$, and $s_{21}=\Sigma_{i=1}^nx_{i2}x_{i1}$.

Mean vector is then: $\mu^{(k)}=(s_1/n, s_2/n)$.

Covariance matrix is: $\Sigma^{(k)}=$$\left( \begin{matrix} \Sigma _{11} & \Sigma _{12} \\ \Sigma _{21} & \Sigma _{22} \\ \end{matrix} \right)$$=$$\left( \begin{matrix} s_{11}/n-(s_1/n)^2 & s_{21}/n-(s_2/n)(s_1/n) \\ s_{12}/n-(s_1/n)(s_2/n) & s_{22}/n-(s_2/n)^2 \\ \end{matrix} \right)$$ $

Other parameters in the question are obtained. Conditional mean and conditional variance of $X_2|X_1$ is calculated by the formula in the reference of the question.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.