I have documents of pure natural language text. Those documents are rather short; e.g. 20 - 200 words. I want to classify them.
A typical representation is a bag of words (BoW). The drawback of BoW features is that some features might always be present / have a high value, simply because they are an important part of the language. Stopwords like the following are examples: is, are, with, the, a, an, ...
One way to deal with that is to simply define this list and remove them, e.g. by looking at the most common words and just deciding which of them don't carry meaning for the given task. Basically by gut feeling.
Another way is TF-IDF features. They weight the words by how often they occur in the training set overall vs. how often they occur in the specific document. This way, even words which might not directly carry meaningful information might be valuable.
The last part is my question: Should I remove stopwords when I use TF-IDF features? Are there any publications on this topic? (I'm pretty sure I'm not the first one to wonder about this question)