2
$\begingroup$

In Latent Dirichlet Allocation (LDA), is it reasonable to reconstruct the original bag-of-words using the document-by-topic and topic-word inferred matrices?

I understand that I will not get frequencies by reconstructing the original matrix, but is the non-zeros after reconstruction valid?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

It is possible to produce a corpus from the learned LDA parameters ($\theta$ and $\phi$) according to the generative model of LDA but it is not realistic to expect that you would recreate the original documents (in bag-of-words form). To be more specific, it is possible - but highly improbable - that you would generate the bag-of-words documents corresponding to the input corpus.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you bogatron, so it is not a good idea to compare two topic models based on how well they 'reconstruct' the original documents, right? can you please explain why? or point me to good (technical) reads? $\endgroup$
    – bma
    Mar 17, 2016 at 7:46
  • $\begingroup$ Are the two topic models both trained from the same "original" corpus? $\endgroup$
    – bogatron
    Mar 18, 2016 at 13:25
  • $\begingroup$ Yes! I have 1 input, 2 variations of LDA, each model learns the two parameters (theta and phi). Is is reasonable to assume that the best model in terms of inferring latent patterns, is the model with the best reconstruction error? $\endgroup$
    – bma
    Mar 19, 2016 at 9:31
  • $\begingroup$ That may be true but then you would be stuck with the problem of defining "reconstruction error". I suspect you would be better off using something like perplexity to compare the learned models. $\endgroup$
    – bogatron
    Mar 24, 2016 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.