# Identify important less frequent words

I have set of newspaper articles. I want to identify important less frequent words in the set of newspaper articles. Currently I am using TF-IDF scores. However, it does not seem to be a good metric in my problem. Is there any better way of doing this?

Thank you in advance :)

• Describe the problem. Non-stationarity? Neologisms? – Emre Nov 27 '17 at 16:16
• @Emre This is a classification of newspaper atricles based on topics. For example sports, crime etc. Iwant to identify the important words to do this. – Volka Nov 28 '17 at 3:34
• You want to classify or tag the article by topic? If so, you want a document classifier or topic model depending on whether the topics are fixed or not. How did you classifiy using the TF-IDF features and how well did it perform? I suppose you tried a bunch of classifiers and did not like the results? Try augmenting the features with the document's topic embedding. – Emre Nov 28 '17 at 4:00