2
$\begingroup$

I try to find the optimal number of topics on a synthetic corpus (so a list of lists of tokens I generate using various parameters). I, therefore, know the true number of topics and the true topics distributions. I believe it is a good way to test unsupervised methods. The issue is that I completely fail to find the correct number of topics.

I am using NMF and LDA from gensim with c_v and u_mass coherence scores. It should be easy game to find the optimal number of topics so I do not use hyperparameters for tuning. I believe the issue is deeper than that.

The code is available here.

It is well documented. The script to run is 'myscript.py'. It uses functions in 'mymodule.py'. You just need to install gensim and pandas (see requirements.txt if needed).

Any thoughts?

Improve this question
$\endgroup$
2
  • $\begingroup$ Welcome to DataScienceSE. Could you please clarify how you generate the synthetic data? I had a quick look at the code and I noticed that you generate token frequency using uniform distribution: I don't know if this is the problem but it's certainly not realistic. In a realistic topic modelling problem the fact that tokens follow a Zipf distrib might play a role. $\endgroup$
    – Erwan
    May 5, 2021 at 21:49
  • $\begingroup$ Thanks Erwan it is the kind of comment I was hoping : I did not know Zipf's law. First topics are created: a topic is a prob distri of tokens (for ex with exponential distri topic_0 = 0.6*'a0'+0.2*'a1'+0.1*'a2'+...). Then docs are generated: a doc is a list of tokens. A doc depends on 2 things: tokens distri within topics (see ex above) and topics distri (for ex doc_0 is 0.7*topic_0+0.2*topic_1+0.1*topic_2). I tried uniform and/or expo distri for tokens distri (=topic distri) and topics distri (=doc distri) but without success. I will implement Zipf distri for tokens distri and see how it goes $\endgroup$
    – seaslug95
    May 6, 2021 at 6:26

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.