I try to find the optimal number of topics on a synthetic corpus (so a list of lists of tokens I generate using various parameters). I, therefore, know the true number of topics and the true topics distributions. I believe it is a good way to test unsupervised methods. The issue is that I completely fail to find the correct number of topics.
I am using NMF and LDA from gensim with c_v and u_mass coherence scores. It should be easy game to find the optimal number of topics so I do not use hyperparameters for tuning. I believe the issue is deeper than that.
The code is available here.
It is well documented. The script to run is 'myscript.py'. It uses functions in 'mymodule.py'. You just need to install gensim and pandas (see requirements.txt if needed).