# Why does my random forest classifier predicts one class more often?

I have a random forest classifier that predicts 0 class about twice as often as class 1. It also predicts class 0 with higher probabilities than class 1.

It is not a imbalanced dataset. I tried setting class 1 weight to 100 and it seems to solve the problem, though I suppose it's not a correct solution :D K-NN gives the same problem. Since I changed y from 0 and 1 to B and A it started to predict second class more frequently. So can the problem be somehow connected to data type?

Code:

dataset = pd.read_csv('regtraining.csv')
X = dataset.iloc[:, :-5].values
y = dataset.iloc[:, 50].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
classifier = RandomForestClassifier()
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

print(confusion_matrix(y_test, y_pred))


When I try multiple random states of train test split, one of predicted classes is always predicted much more frequently.

Edit: After some research I think the random forest splits so that the classes are predicted in an alphabetic order (A, B values for y give more of A but B, A values for y give more of B).

• I am not concerned with a class being predicted more than another. I am concerned with what makes the "best" predictions. In this case there is a cutoff value that is being used to make the prediction. Is that cutoff value optimal for your problem? It needs to balance the costs of false positive/negative with the benefit of true positive/negative. Changing the cutoff value will change the predicted class. In your case, what are the best metrics for your problem and are the current prediction optimizing that metric? Commented Jan 20, 2021 at 10:02
• Might be because of data. Could you share the confusion matrix. Also, the split diagram with a Tree of depth 1. This might have some clue Commented Jan 21, 2021 at 13:56
• use a decision tree with the random forest and graph out the rules. you can then see what tree ensembly is causing the classification to occur more frequently Commented Jul 25, 2022 at 14:57

While prediction, If you have more date relevant to class 0 obviously this is the expected one.

But if your observation is like, the model predicts more data with class 0 as incorrect, then you model is kind of over-fitted.

To avoid these,

• 57% and 43% is not kind of class imbalanced, but still you use class_weight with model.

• Feature Engineering helps a lot with these kind of situations.

• Hi, thanks for your answer. I don't think overfitting is the problem, as I lowered max depth to 1 and it's still the same. Commented Jan 20, 2021 at 9:52

It might be overfitting case. You can perform hyperparameter tuning with GridSearchCV or RandomizedSearchCV. Then you can check the performance of the model. If you still have bias in the prediction, you much do feature engineering and feature selection of the data.