# Need help in improving accuracy of text classification using Naive Bayes in nltk for movie reviews

I am referring http://www.nltk.org/book/ch06.html for generating a movie review classifier. It considers all words (Nouns, adjectives, verbs..) as part of feature set. I am trying to build a classifier that considers only the verbs and assesses if the movie review is positive or negative.

Please explain if this approach is better, if yes how can it be improved else what other parts of speech tags need to be included to improve the feature set.
Please refer to the following code
1)Import nltk and corpus
2)Understand the categories
3)Create a list of "documents" where each document consists of words that are not in stopwords.words() and filtered from list of "punctuations" as well.
4)Shuffle the documents and generate a list called "all_words" that contains all the words appearing in movie_reviews minus stopwords and punctuations.
5)Create a Freq Dist of "all_words"
6)Create a list of "verbs" by looking at the pos_tag of each word appearing in "all_words"
7)Create a feature dictionary for each document in the "documents" list created in step 3, such that the dictionary will contain keys corresponding to words in "verb" list and a boolean to show if that verb appeared in the document under consideration. This is handled by documentFeature().
8)Create a naive bayes classifier instance and train and compute test accuracy.

from nltk.corpus import movie_reviews
from nltk.corpus import stopwords
from nltk.tokenize import RegexpTokenizer
import nltk
movie_reviews.categories()
#['pos','neg']
#the regextokenier is used to tokenize the words
tokenizer=RegexpTokenizer(r'\w+')
#creating documents based on filtration of stopwords for each review and running a tokenizer on each document
documents=[(tokenizer.tokenize(' '.join(set(i for i in movie_reviews.words(fileid))-set(stopwords.words()))),category)
for category in movie_reviews.categories()
for fileid in movie_reviews.fileids()
]
import random
random.shuffle(documents)
#each document contains words that are not in stopwords and punctuations.
for i in documents[:5]:
temp=nltk.FreqDist([j.lower() for j in i[0]])
print(temp.most_common(5),i[1])
#output of 5 documents
#[('vampires', 1), ('clever', 1), ('interesting', 1), ('sunlight', 1), ('partners', 1)] neg
#[('family', 1), ('nino', 1), ('friends', 1), ('acting', 1), ('higher', 1)] pos
#[('inconsistent', 1), ('eye', 1), ('yes', 1), ('interesting', 1), ('praise', 1)] neg
#[('acting', 1), ('science', 1), ('bucks', 1), ('huge', 1), ('terrific', 1)] pos
#[('acting', 1), ('shielded', 1), ('somewhere', 1), ('think', 1), ('touched', 1)] neg

#generate a list called 'all_words' that contains all the set of words that have appeared so far
all_words=tokenizer.tokenize(' '.join(set(i for i in movie_reviews.words())-set(stopwords.words())))
freqdist=nltk.FreqDist(all_words)

#create a list of all verbs for each word appearing in 'all_words'
verb=[]
pos_=nltk.pos_tag(all_words)
#print([i[1] for i in pos_])
for i in pos_:
if i[1] in ['VB','VBG','VBN','VBZ','VBD','VBP']:
verb.append(i[0])

#document - feature set, build a dictionary of verbs for each document
def documentFeature(document):
feature={}
for i in verb:
feature['contains({0})'.format(i)]=(i in document)
return feature
#build a naive bayes classifier
featureSet=[(documentFeature(d),c) for d,c in documents]
trainSet,testSet=featureSet[100:], featureSet[:100]
classifier=nltk.NaiveBayesClassifier.train(trainSet)

print(nltk.classify.accuracy(classifier, testSet))
#0.03 a very poor accuracy on the testset


• 0.03% accuracy seems way too low even if verbs turn out to be a very bad feature. Maybe you have some bug or you are reading too few documents? – Miguel Dec 30 '17 at 11:52
• Two questions: 1. Why Naive Bayes? 2. Why just the verbs? Starting out with these limitations is basically a form of researcher bias. Why not maintain an open mind to which models will work best with any number of possible inputs? Besides, for NLP there are many, many more algorithms that work better than Naive Bayes. – I_Play_With_Data Feb 28 '18 at 14:58
• Can you please provide a link to the data so that we can help you? – JahKnows Oct 27 '18 at 2:14