I'm currently training a random forest classifier with 1000 trees and am receiving the following confusion matrix. I've split the training set 75/25 against the test set but am not sure why I'm seeingf such a low True Negative score? This is a binary classification problem and my dataset contains 88% of Class 0 whereas Class 1 represents 12% of the total content. I've tried experimenting with class_weights but I have not been able to get the True Negative score above 6%. What am I missing as it seems to pass most tests including Cross-Validation > .90 etc.

enter image description here

  • $\begingroup$ You hardly beat the naive baseline of 88% accuracy. So your model is simply not very well suited for your problem or the data are just very hard to predict. However, with the given information it is impossible to say what could be done to improve the fit. $\endgroup$ – Peter Oct 16 at 21:33

From your confusion matrix it actually looks like your class imbalance is over-favoring the positive class (90.04% to 9.96%). Because you have such an extreme class imbalance in your data your random forest is likely just classifying almost everything as positive class and calling it a day.

To solve this issue you need to either down-sample your larger class or up-sample your lesser class to reach closer to 50/50 split. Otherwise you will likely just end up with a classifier that grossly over-predicts the positive class.

When you are splitting your data you can also try stratifying by your label (so that the same proportion of positive to negative class exists in your training and testing sets).

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.