1
$\begingroup$

I'm really new to Data Science and text mining. I want to build a relevancy scoring model. Suppose I have a bag of words (guns, military, terrorists). I also have a list of articles. I want to find if the articles are relevant to the bag of words.

What I have thought of doing:

  1. clean the article by removing stop-words, punctuation and stemming.
  2. Try to find the average score of each word of the article with the bag of words using wordnet.wup_similarity
  3. Then compare the scoring of the articles. If they pass a threshold, they will be considered relevant, else not.

I have also seen TF-IDF and topic modelling and then I can find the semantic similarity between the topics identified and the bag of words and see if relevant.

But this might be a very naive way of doing it. Can anyone suggest any methods/papers/algorithms to perform the same?

Any help is appreciated. Thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.