5
$\begingroup$

am going to do topic modeling via LDA. I run my commands to see the optimal number of topics. The output was as follows: It is a bit different from any other plots that I have ever seen. Do you think it is okay? or it is better to use other algorithms rather than LDA. It is worth mentioning that when I run my commands to visualize the topics-keywords for 10 topics, the plot shows 2 main topics and the others had almost a strong overlap. Is there any valid range for coherence?

Many thanks to share your comments as I am a beginner in topic modeling.

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

LDA being a probabilistic model, the results depend on the type of data and problem statement. There is nothing like a valid range for coherence score but having more than 0.4 makes sense. By fixing the number of topics, you can experiment by tuning hyper parameters like alpha and beta which will give you better distribution of topics.

The alpha controls the mixture of topics for any given document. Turn it down and the documents will likely have less of a mixture of topics. Turn it up and the documents will likely have more of a mixture of topics.

The beta controls the distribution of words per topic. Turn it down and the topics will likely have less words. Turn it up and the topics will likely have more words.

The main purpose of lda is to find hidden meaning of corpus and find words which best describe that corpus.

To know more about coherence score you can refer this

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.