How to select features for Text classification problem

Question

I am working on a problem where we need to classify user query into multiple classes.

Problem:

Suppose we are running a website for selling products. The website has a form where the user can write any complaint or issue.

In order to resolve users issue, we thought to classify issues into predefined classes so that we can understand what type of problems users are facing.

Issues classes can be like

Class 1: Payment Issues
Class 2: Registration issues
Class 3: Order booking issues
Class 4: Problem accessing website app
...

I am thinking to apply random forest but cannot decide what features should I select. Any suggestions would be of great help thank you

Answer 1

Since I am not sure where you are at the moment in your solution, let me give you a comprehensive, yet brief, view of what you should be looking to do.

1. Preparing the training data: You would be required to collect data with the correct classifications. Once you have this, you should incorporate text mining algorithms (bag of words for instance) and end up with a document-term matrix (or term-document matrix, whichever is appropriate). Think of this as a list of all the different words (or word collections) possible in a complaint. These would in turn serve as your features
2. Training the data: This is where the algorithms you have stated above (RandomForest for instance) come into picture. I would also suggest you to try other boosting algorithms here subject to data availability
3. Testing on a holdout sample: Test the model accuracy on a holdout sample and finalize on the models you would like to choose. Note that you might also have to tune the text mining algorithm in case you do not get appropriate features for your classification algorithm
4. Classifying new complaints: Apply the text mining algorithm to the newer data to generate the features as before and run it by the classifier

Hope this gives you a good starting point for building your text classification solution.

Answer 2

The basic way:

1. Tokenize text into list of words.
2. Stem words to obtain 'root forms'.
3. Decide how many most frequent words you want to consider / what is the minimum count of words that matter.
4. Skip the stop words - most frequent ones ('a', 'the') / closed grammar classes like 'and', 'but'.
5. You get say 300 most frequent words which matter.
6. Now the easiest way is to use calculate tf-idf for each word for each document and you have 300 features for each document.