I am working on a simple text classification problem where I have as inputs tweets and as class whether that tweet contains fake news or not (0 is real news, 1 is fake news). I have trained a logistic regression model on some training data with labels and then I wanted to predict the label of new unseen data and also print the weights of each feature in order to explain the prediction. The way I feed the data to the logistic regression algorithm is by using a count vectorizer. I have done this for many tweets and I noticed something I find odd and that I can't explain.

For example given this tweet "Thanks @Lyricoldrap and @BridgetteWest we got this covid beat! 👊👊soon!! Y'all are a blessing! Soup, crackers and more! 🤗😘 https://t.co/TKnOspiqaj", the logistic regression predicts the class 0, with probability 0.918.

However, when I print the weights of the features for this prediction I get the following:

features   weights     words
148       0.265943      soon
40        0.217010     covid
66        0.181601       got
18       -0.015743  blessing

This intuitively tells me that the word "soon" contributed the most to the prediction towards class 1 with the biggest weight, followed by "covid" and "got" and finally "blessing" contributed to the class of 0 (since it has a negative weight). Is this intuition correct? If it is not correct how is it possible I get as a prediction the class 0 with such high probability if three of my most important features are pushing the prediction in the opposite label? Or am I missing something here?

  • $\begingroup$ After posting this, I realised that I forgot to add the intercept's value which of course also contributes to the prediction.. $\endgroup$
    – thelaw
    Feb 20 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.