0
$\begingroup$

In the setting of Variational AutoEncoders, i.e. when we want to find the posterior distribution

over the data generating, latent variable z, given some observations x, what exactly (which part of the equation) makes this posterior distribution intractable and why?

Cheers

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

It's usually the denominator $p(x)$ (the "evidence") which is intractable. You could attempt to compute it by marginalizing over the latent variable $p(x) = \int p(x|z)p(z)dz$. However, you would need to evaluate all possible values of $z$ which would require exponential time. (That's why in maximium-likelihood estimation you have no intractability problem because you can treat the evidence as a constant.)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer. Might sharing some more insights on the "exponential time" part? $\endgroup$
    – Matt
    Aug 28, 2020 at 14:58
  • 1
    $\begingroup$ $z$ will usually be high-dimensional and to accurately sample from the large" volume of the space" you need exponentially many more samples for each additional dimension. ("curse of dimensionality"). even if there is a closed-form solution to the integral the computation can still be exponential see e.g. the example here $\endgroup$
    – oW_ ♦
    Sep 9, 2020 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.