My main concern is that i need to understand that how does the random forest do majority voting in scikit learn source code. I did not find that specific code in source code of RandomForest. if anybody knows, then kindly explain that. Thanks
To quote from the documentation:
The predicted class probabilities of an input sample are computed as the mean predicted class probabilities of the trees in the forest. The class probability of a single tree is the fraction of samples of the same class in a leaf.
The predictions are then simply the class with the highest average class probability.
You can find the source code here.
Note that this is different from classical majority voting which is usually understood to be the most common class prediction among trees whereas here the voting happens on the class probability level.
The source code for regression is here:
The predicted regression target of an input sample is computed as the mean predicted regression targets of the trees in the forest.
2$\begingroup$ +1; to emphasize, sklearn's random forests do not use "majority vote" in the usual sense. $\endgroup$– Ben Reiniger ♦Oct 24, 2019 at 18:04
$\begingroup$ Agree, and kindly suggest to edit the answer to explicitly point this out $\endgroup$ Oct 25, 2019 at 9:19
1$\begingroup$ Done. Thanks for the feedback. $\endgroup$– oW_ ♦Oct 25, 2019 at 12:38
A Random Forest is an ensemble of decision trees. Each decision tree will reach a "conclusion" (i.e., a prediction) about each observation. All trees are then combined together.
What does it mean?
- if you are training a Random Forest regressor, this combination is an average of each tree's prediction.
- if you are training a Random Forest classifier, each tree's classification is combined into a final classification through a "majority vote" mechanism.
$\begingroup$ Please notice that what OP asks is "how does the random forest do majority voting in scikit learn" (emphasis mine). $\endgroup$ Oct 25, 2019 at 9:17