In the setting of Variational AutoEncoders, i.e. when we want to find the posterior distribution

over the data generating, latent variable z, given some observations x, what exactly (which part of the equation) makes this posterior distribution intractable and why?



It's usually the denominator $p(x)$ (the "evidence") which is intractable. You could attempt to compute it by marginalizing over the latent variable $p(x) = \int p(x|z)p(z)dz$. However, you would need to evaluate all possible values of $z$ which would require exponential time. (That's why in maximium-likelihood estimation you have no intractability problem because you can treat the evidence as a constant.)

  • $\begingroup$ Thanks for your answer. Might sharing some more insights on the "exponential time" part? $\endgroup$ – Matt Aug 28 '20 at 14:58
  • $\begingroup$ $z$ will usually be high-dimensional and to accurately sample from the large" volume of the space" you need exponentially many more samples for each additional dimension. ("curse of dimensionality"). even if there is a closed-form solution to the integral the computation can still be exponential see e.g. the example here $\endgroup$ – oW_ Sep 9 '20 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.