What is the most efficient method for detecting whether the article is mostly about a specific topic, but without lots of data for training? My task is to determine how much a document is e.g. about the weather or holidays or several other specific topics.

I was looking towards LDA and TFIDF but from what I understand this approach is unsupervised and works well for clustering/grouping large numbers of documents based on vocabulary frequency. These techniques have a limitation in terms of controlling what topics the algorithm should focus on. Additionally, in my case, I do not have a lot of data to train the model on. So I was thinking about generating lists of tokens characteristic to some specific topics and then measuring cosine similarity with word2vec between the vocab used in the document with the list of target tokens.

My questions are:

  1. is it the right way forward or there are better ways of achieving this?
  2. How the final score should be calculated - is an average of similarities between tokens okay? I am afraid that for e.g. if I create 100 target tokens per topic, the similarities will somehow cancel out yielding similar scores.
  3. What I like about LDA is that it shows the probability levels per multiple topics. Is there an algorithm similar to LDA, where I could seed the topics rather than merely stipulate the number of clusters?

1 Answer 1


Topic Modeling

Topic modeling is a technique for taking some unstructured text and automatically extracting its common themes, it is a great way to get a bird's eye view of a large text collection. simply, it is a type of statistical model for discovering the abstract " topics" that occur in a collection of documents

Popular topic modeling algorithms

Evaluating Model

The final score can be calculated using the below metrics:

  1. Perplexity - This is a statistical measure of how well a probability model predicts a sample. As applied to LDA, for a given value of k, you estimate the LDA model. Then given the theoretical word distributions represented by the topics, compare that to the actual topic mixtures, or distribution of words in your documents. - Lower the better.
  2. Coherence Score - Is defined as the average/median of the pairwise word-similarity scores of the words in the topic - The higher the better.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.