Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Let's say I want to classify if the employee will churn or not. In my random forest, I have 6 estimators where 3 of them predict the employee to churn and the other estimators predict the employee to retain. In short, 3 estimators = predicted to churn and 3 estimators = predicted to retain.
I know that the random forest works by getting the most votes for each class or by getting the mode.
The question is how does the random forest predict in this case?
Seems like it predicts the first class. Sklearns Random Forest implementation generates probabilities for each class by averaging the probability predicted by each estimator into an array of shape (n_samples, n_classes), and then uses np.take(np.argmax()) to select the highest probability, something akin to this:
# Pretend "a" is our averaged predictions for the forest. So the first sample is predicting 78% probability
class 0, 22% class 1. The second has the probabilities reversed and the third is 50/50 split.
a = np.array([[0.78, 0.22], [0.22, 0.78], [0.5, 0.5]])
np.argmax(a, axis=1)
The output is array([0, 1, 0], dtype=int64). These are the indices of the highest value in each sample of the array, and for the sample with an even split you can see it's picking class 0.
edit:
If you want to see it yourself, the relevant piece of code is line 540 in forest.py of sklearn:
https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/ensemble/forest.py
