0
$\begingroup$

Does anyone have a good idea for how to compare topic modeling done by NMF and LDA? Let's say I fit LDA to a dataset and generate topic-word and document-topic distributions--I can use perplexity, for instance, to measure the goodness of fit of the model. If I fit NMF to the same dataset and generate topic-word and document-topic distributions, I can take the resultant fitted data as $X^{Est} = HW^T$ and use $\|X^{Original} - X^{Est}\|_{Fro}$ as a goodness of fit of the model.

What is an objective metric that I can use to compare the goodness of fits to each other? Can either of these metrics be used for the other model?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
3
$\begingroup$

NMF and LDA models produce topic-word and document-topic distributions, so you can compare these models on evaluation tasks of topic coherence (i.e. evaluating topic-word distributions), document clustering/classification (i.e. evaluating document-topic distributions) or information retrieval (i.e. evaluating topic-word and document-topic distributions together).

See some examples in "Improving topic models with latent feature word representations", "LDA-Based Document Models for Ad-hoc Retrieval".

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the cursory - but useful - connection of topic coherence to topic-word, doc clustering/classification to evaluation of doc-topic and IR to evaluation of topic-word and doc-topic together. I had not seen that presented before. $\endgroup$
    – WestCoastProjects
    Dec 14, 2017 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.