1
$\begingroup$

Let's say I want to classify if the employee will churn or not. In my random forest, I have 6 estimators where 3 of them predict the employee to churn and the other estimators predict the employee to retain. In short, 3 estimators = predicted to churn and 3 estimators = predicted to retain.

I know that the random forest works by getting the most votes for each class or by getting the mode.

The question is how does the random forest predict in this case?

$\endgroup$
1
$\begingroup$

Seems like it predicts the first class. Sklearns Random Forest implementation generates probabilities for each class by averaging the probability predicted by each estimator into an array of shape (n_samples, n_classes), and then uses np.take(np.argmax()) to select the highest probability, something akin to this:

# Pretend "a" is our averaged predictions for the forest. So the first sample is predicting 78% probability
class 0, 22% class 1. The second has the probabilities reversed and the third is 50/50 split. 

a = np.array([[0.78, 0.22], [0.22, 0.78], [0.5, 0.5]])

np.argmax(a, axis=1)

The output is array([0, 1, 0], dtype=int64). These are the indices of the highest value in each sample of the array, and for the sample with an even split you can see it's picking class 0.

edit:

If you want to see it yourself, the relevant piece of code is line 540 in forest.py of sklearn:

https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/ensemble/forest.py

$\endgroup$
  • $\begingroup$ Thank you so much! I already got it. Scikit-learn modules did not follow the real theory of random forest. :) $\endgroup$ – King Jul 31 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.