Why does my random forest classifier predicts one class more often?

Question

I have a random forest classifier that predicts 0 class about twice as often as class 1. It also predicts class 0 with higher probabilities than class 1.

It is not a imbalanced dataset. I tried setting class 1 weight to 100 and it seems to solve the problem, though I suppose it's not a correct solution :D K-NN gives the same problem. Since I changed y from 0 and 1 to B and A it started to predict second class more frequently. So can the problem be somehow connected to data type?

Code:

dataset = pd.read_csv('regtraining.csv')
X = dataset.iloc[:, :-5].values
y = dataset.iloc[:, 50].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
classifier = RandomForestClassifier()
classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

print(confusion_matrix(y_test, y_pred))

When I try multiple random states of train test split, one of predicted classes is always predicted much more frequently.

Edit: After some research I think the random forest splits so that the classes are predicted in an alphabetic order (A, B values for y give more of A but B, A values for y give more of B).

Thanks for your answers, I am new to machine learning :D

Answer 1

While prediction, If you have more date relevant to class 0 obviously this is the expected one.

But if your observation is like, the model predicts more data with class 0 as incorrect, then you model is kind of over-fitted.

To avoid these,

• 57% and 43% is not kind of class imbalanced, but still you use class_weight with model.

• Feature Engineering helps a lot with these kind of situations.

Answer 2

It might be overfitting case. You can perform hyperparameter tuning with GridSearchCV or RandomizedSearchCV. Then you can check the performance of the model. If you still have bias in the prediction, you much do feature engineering and feature selection of the data.