As my title already says: can the ELBO be negative?

$ELBO_\lambda = KL[q_\lambda(w)||P(w)] - \mathbb{E}_q[\log P(\mathcal{D|w})] $

Can I theoretically adjust my prior $P(w)$ such that $KL=0$ and also $\mathbb{E}_q>0$? If yes, why call it a lower bound on evidence? Since $P(\mathcal{D}) \geq 0$ per definition and a negative lower bound has no information.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.