I'm really new to Data Science and text mining. I want to build a relevancy scoring model. Suppose I have a bag of words (guns, military, terrorists). I also have a list of articles. I want to find if the articles are relevant to the bag of words.

What I have thought of doing:

  1. clean the article by removing stop-words, punctuation and stemming.
  2. Try to find the average score of each word of the article with the bag of words using wordnet.wup_similarity
  3. Then compare the scoring of the articles. If they pass a threshold, they will be considered relevant, else not.

I have also seen TF-IDF and topic modelling and then I can find the semantic similarity between the topics identified and the bag of words and see if relevant.

But this might be a very naive way of doing it. Can anyone suggest any methods/papers/algorithms to perform the same?

Any help is appreciated. Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.