5
$\begingroup$

I have one corpus of documents on diabetes, another on Leonardo da Vinci, and another on animation and computer graphics. I combined all of these documents into a LDA and got a topic like the one below. I'm listing the top 30 terms, in descending order.

TOPIC 83: ['produced', 'company', 'john', 'weight', 'motion', 'information', 'painting', 'original', 'training', 'people', 'health', 'life', 'jesus', 'feature', 'body', 'lucretia', 'computer', 'graphics', 'time', 'madonna', 'story', 'florence', 'type', 'animated', 'paul', 'diabetes', 'animation', 'exercise', 'peter', 'film']

Many of these words do not co-occur with each other in any documents. For instance, 'lucretia' does not co-occur with 'exercise'. Yet how are these terms put together in the same topic, in the top 30 words? (30 out of 20K words or so)

Perhaps it has to do with my particular implementation? I'm using the gensim library for Python. Or is it a flaw of LDA generally?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

It could be partially due to the number of topics you selected but the fact that two words rank high for a given topic doesn't necessarily mean that the two words will frequently occur in the same document. Consider a topic that generally corresponds to "medicine". You could have a number of medical documents associated with heart disease and a number of medical documents associated with injuries. They may all be linked by common terms such as "diagnose", "treatment", "patient", etc. But you wouldn't necessarily expect that words like "heart" and "aorta" would appear in documents containing words such as "sprain" and "fracture", even though they are all associated with the general topic of medicine. If one were to choose a larger number of topics, those words might be separated into more specific topics (e.g., "heart disease" and "sports injuries").

Improve this answer
$\endgroup$
2
$\begingroup$

It is a feature (I won't call it flaw) of LDA generally. Usually, you get some very clearly defined topics and some very muddy topics (best termed "Miscellaneous" or so).

I also advice not to overinterprete the topics from topic modelling. Just changing a little (e.g. the random seed, taking a few texts out, applying some "small" modifications to the corpus) changes a significant amount of the topics.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Interesting comments in your second paragraph. Are there any general rules of thumb in this regard? Or guidelines in how certain parameter choices or data cleaning will affect your results? $\endgroup$
    – Matt
    Aug 3, 2015 at 14:28
  • $\begingroup$ @Matt: I'd like to know that, too. I just took some data I understand already pretty well and experimented with them, with the results above. I found that adjusting the stopwords to the actual corpus helps a lot in clearer topic definition. $\endgroup$
    – user10169
    Aug 3, 2015 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.