5
$\begingroup$

I am trying to understand exactly how the meta random forest classifier determines the final prediction, I understand that there is a voting system and an aggregation from the decision trees is used to find the final prediction, I have read from here: Classification Random Forests in Python:

"Random forest is an ensemble decision tree algorithm because the final prediction, in the case of a regression problem, is an average of the predictions of each individual decision tree; in classification, it's the average of the most frequent prediction"

I read in the RFC source code: Ensemble/Forest:

"The predicted class of an input sample is a vote by the trees in the forest, weighted by their probability estimates. That is, the predicted class is the one with highest mean probability estimate across the trees"

Does this mean that for example:

Decision tree example

We have 3 Classes (A, B and C) with 4 estimators, each tree has a certain amount of final leaf nodes with a class prediction;

Probabilities of Class A, B or C are a result of the number of the class prediction from the leaf nodes, i.e Tree1: Class A is predicted 3 times out of 10 possible times, the other 7 times a different class was predicted.

Class A: [0.30, 0.25, 0.30, 0.25] | mean = 0.275
Class B: [0.10, 0.40, 0.10, 0.40] | mean = 0.250
Class C: [0.70, 0.65, 0.70, 0.65] | mean = 0.675

So the Meta Classifier would predict Class C as it has the highest mean probabilities. Is this correct? Or am I looking at this completely the wrong way?

$\endgroup$

1 Answer 1

4
$\begingroup$

Each data point $x$ that you are predicting necessarily only lands in one leaf node for each tree. So if I have N trees, and $K_A$ is the number of trees in which $x$ ends up in a leaf node which predicts A, then we can say $x$ is class $A$ with "probability" $K_A/N$, and similarly for other classes. Then you can just make an argmax prediction for which class $x$ belongs to based on the "probabilities" (I keep putting this in quotes because it isn't an actual probability, but it's sometimes ok to think of it this way.)

$\endgroup$
3
  • $\begingroup$ Thank you for your response, I am still having a little difficulty understanding something. From what I understand from your explanation; for each data point 𝑥 is set to be class A if the probability that 𝑥 appears most frequently as Class A in the the trees, how then can we determine which tree the final aggregation is derived from? $\endgroup$ May 6, 2019 at 11:45
  • 1
    $\begingroup$ The final prediction is a vote based on every tree in your forest. You run your data point through every tree, see how each tree votes, and use the percentage of how many trees voted for each class as "probabilities." Is the question here "how can I look at WHICH trees voted for which class?" $\endgroup$
    – Andy M
    May 6, 2019 at 13:32
  • $\begingroup$ Also, this answer is a clear pythonic explanation for prediction estimation. $\endgroup$
    – Mario
    Feb 19 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.