My dataset has around 1000 features and 30k rows.
All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data points, 9k "Medium" data points and 20K "Large" data points. I want to find feature importance. I am not sure which algorithm will work best for features with only 1s and 0s. I tried Chi2 but accuracy is very low:
#find best scored 10 features select_feature = SelectKBest(chi2, k=10).fit(X_train, Y_train) X_train_2 = select_feature.transform(X_train) X_test_2 = select_feature.transform(X_test) clf_rf_2 = RandomForestClassifier(n_estimators=1000, n_jobs=-1, random_state = 100) clr_rf_2 = clf_rf_2.fit(X_train_2,Y_train) ac_2 = accuracy_score(Y_test,clf_rf_2.predict(X_test_2)) print('Accuracy is: ',ac_2) cm_2 = confusion_matrix(Y_test,clf_rf_2.predict(X_test_2))
My accuracy is only 58% and the model is only good in predicting "Large" data points. Below is the Confusion Matrix
Small Medium Large Small 6 17 3000 Medium 9 9 2000 Large 31 32 7500
I mostly have 2 questions:
- I am not sure which algorithm will work for best features with only 1s and 0s
- How do I get higher accuracy using chi2 (predict Small and Medium class better)
Thanks! I would really appreciate any input on this.