I'm learning about the variational inference - mean field approximation on from this online course DeepBayes2019 page 30

The probabilistic model is written as follows: $p(X, Z \mid \pi, \mu, \lambda) = Dir(\pi \mid \alpha)\prod_{i=1}^{N} \prod_{k=1}^{K} [\pi_k \mathcal{N}(x_i \mid \mu_k, \lambda_k^{-1})]^{z_{ik}}$

The E-step is defined as $p(Z, \pi \mid X, \mu, \lambda) \approx q(Z, \pi) = q(Z) q(\pi)$ with

$$ q(Z) = \prod_{i=1}^{N}q(z_i) \quad q(z_i) = \prod_{k=1}^{K} \rho_{ik}^{z_{ik}} / \sum_{k=1}^{K} \rho_{ik} \\ q(\pi) = Dir(\pi \mid \alpha'), \quad \alpha_k' = \alpha_k + \sum_{i=1}^{N} \mathbb{E}_{q(Z)}z_{ik} $$

And the M-step is: $$ \mu_k = \frac{\sum_{i=1}^{N} \mathbb{E}_{q(Z)}z_{ik}x_i}{\sum_{i=1}^{N} \mathbb{E}_{q(Z)}z_{ik}}, \quad \lambda_k = \frac{\sum_{i=1}^{N}\mathbb{E}_{q(Z)}z_{ik}}{\sum_{i=1}^{N}\mathbb{E}_{q(Z)}z_{ik}(x_i - \mu_k)^2} $$

  1. My problem is that I don't understand how they defined the $\rho$ in the proof they wrote: $$ log q(Z)= \sum_{i=1}^{N} \sum_{k=1}^{K} z_{ik}(\mathbb{E}_{q(\pi)} log \pi_k + log\mathcal{N}(x_i \mid \mu_k, \lambda_k^{-1}))+ cst \\ = \sum_{i=1}^{N} \sum_{k=1}^{K} z_{ik} log \rho_{ik} + cst $$

  2. I am not sure that I understand the role of the $z_{ik}$ matrix


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.