How to improve Naive Bayes?

Question

I am solving a problem that address this question "What are the Actions that lead to high or low score?"

I have the following Data that consist of text and score , I want to derive the words or Actions from text that lead to high/low score

I have huge data , and text length can be a paragraph, sample data like below

+----------------------+-------+
|         Text         | Score |
+----------------------+-------+
| Support Team Goal    |    90 |
| Generate Lead        |    80 |
| Contact 30 customers |    30 |
| Support Team Goal    |    30 |
+----------------------+-------+

Approach followed: I followed Naive Bayes way.

1. First I classified my Data into High (score(75 and above)) and Low (score below 75)
2. I converted the High to Term Document Matrix and the Low as well
3. I found words appearing only in low and judge they lead to low score (include word if frequency greater than 7 lets say)
4. I found words appearing only in high and judge they lead to high score (include word if frequency greater than 7 lets say in high)
5. I found probability of common words, For words appearing in both high or low, I calculated the probability of the word occurrence to happen in low/high (include word if probability greater than 75%)

Note: I worked with bigrams. Not sure if this is the right approach to analyze my problem, or there are better techniques. Please advise

Answer 1

Without looking at the actual data, all we can really do is second guess and suggest best practices. Here are some pointers you can pursue -

• Gather more data - If it can be done, nothing like it.
• Improve data quality - The algorithm will always be as good as the data. Try extensive cleaning methods like -
• Lowercase (basic) so all your data is standardized
• Stemming / Lemmatization - These techniques reduces a word to its root form
• Try parts of speech tagging (POS) and retain important parts of speech like nouns and look at their overall importance
• Stopword removal - Just like you tried doing word frequencies, there are popular word sets like the word the that don't help much. They are filtered out
• Correcting Spelling mistakes - If you are expecting there will be spelling mistakes, might be a good idea to correct them
• Converting the unstructured data - I see that you have used bigrams. Try using unigrams and trigrams as well, or in combinations, run your algorithm and see which one works better. Try CountVectorizer, TfidfVectorizer and other techniques like word embeddings as well
• Algorithms - Finally focus on the algorithm itself. For Naive Bayes, focus on MultinomialNB. Try RandomForestClassifierand other ensemble family algorithms. Try Deep Learning techniques with keras. Fine tune hyperparameters based on the validation results

Of course, there are other best practices like splitting your data into train, test and cross validation sets. Evaluating the right metrics such as accuracy, precision, recall and the confusion matrix.

Hope this helps!