I am working on a simple text classification problem where I have as inputs tweets and as class whether that tweet contains fake news or not (0 is real news, 1 is fake news). I have trained a logistic regression model on some training data with labels and then I wanted to predict the label of new unseen data and also print the weights of each feature in order to explain the prediction. The way I feed the data to the logistic regression algorithm is by using a count vectorizer. I have done this for many tweets and I noticed something I find odd and that I can't explain.
For example given this tweet "Thanks @Lyricoldrap and @BridgetteWest we got this covid beat! 👊👊soon!! Y'all are a blessing! Soup, crackers and more! 🤗😘 https://t.co/TKnOspiqaj", the logistic regression predicts the class 0, with probability 0.918.
However, when I print the weights of the features for this prediction I get the following:
features weights words
148 0.265943 soon
40 0.217010 covid
66 0.181601 got
18 -0.015743 blessing
This intuitively tells me that the word "soon" contributed the most to the prediction towards class 1 with the biggest weight, followed by "covid" and "got" and finally "blessing" contributed to the class of 0 (since it has a negative weight). Is this intuition correct? If it is not correct how is it possible I get as a prediction the class 0 with such high probability if three of my most important features are pushing the prediction in the opposite label? Or am I missing something here?
