I used CountVectorizer and TfidfVectorizer seperately to vectorize text which is 100K reviews and passed the vector data to a Decision tree Classifier. Upon using _feature_importances__ attribute of Decision tree Classifier, the feature importance values for all my feature are just 0.0. But with the same dataset, I'm able to find feature importances for naives bayes and logisitic regression by using feature_prob for naive bayes and coef_ attributes for logisitic regression.
Other things I tried: 1. I tried changing ngram_range in countvectorizer 2. I tried limiting/not limiting min_df and max_feature parameters passed in countvectorizers But couldn't make it work.
Any help is appreciated
Code:
positivereviews = df[df.Score == 1]
negativereviews = df[df.Score == 0]
countvect = CountVectorizer(stop_words='english')
positivebow = countvect.fit(positivereviews.CleanedText[0:100000])
pos_xtrain = positivebow.transform(positivereviews.CleanedText[100000:200000])
pos_y = positivereviews.Score[100000:200000]
clf = DecisionTreeClassifier(max_depth=3, min_samples_split=2)
clf.fit(pos_xtrain, pos_y)
def show_most_informative_features(vectorizer, clf, n=20):
feature_names = vectorizer.get_feature_names()
coefs_with_fns = sorted(zip(clf.feature_importances_, feature_names))
top = coefs_with_fns[:-(n + 1):-1]
for (coef_1, fn_1)in top:
print ("\t%.4f\t%-15s" % (coef_1, fn_1))
Counter(clf.feature_importances_)
Counter({0.0: 63514})
1) positivereviews is the dataframe which has features CleanedText and Score where CleanedText is text which is preprocessed and Score is either 1 in this set since I splitted positive and negative review using Score
2) I also searched online for this problem, but couldn't find any instance of this issue
EDIT1: can it be because since we are dealing with categorical feature in this problem, I might be overfitting badly and hence I do not see any value for any features?
Thanks in advance!!