Does Random Forest Regressor use subset of trees to predict value from given data sample?

Question

I will try to draw a little context to my question from title.

I build a Random Forest Regressor from 1000 trees using sklearn. Then I exported all the decision paths along with predicted values for each Path and tree (around 50k decision paths, as one tree includes many decision paths).

I want to predict value for a particular sample. I can't use sklearn method predict() for some reasons but I have its value for comparison. I expected that I will take all decision paths for which the data from my sample meet conditions and calculate average to get final predicted value.

For example, if 3 decision paths corresponds to values from my sample, I could calculate average from 3 predicted values to get the final one.

But result from this operation differs a lot from value obtained by predict() method.

I would be grateful if someone could explain how this regressor works. And the second question is, is there any other way to obtain predicted value based only on decision paths of trees in the regressor.

Thank you very much!

And sorry for not attaching any code. I can clarify the description more if needed :)

Answer 1

The Random Forest Regressor works by creating a multitude of decision trees at training time and outputting the mean prediction of the individual trees at prediction time.

When you use the predict() method, it's not just taking the average of the predictions of the decision paths that your sample meets the conditions for. It's taking the average of the predictions of all the trees in the forest. Each tree in the forest is making a prediction for your sample, regardless of whether the sample meets the conditions for a particular decision path in that tree.

So, if you want to replicate the predict() method using only the decision paths, you would need to do the following:

1. For each tree in the forest, find the decision path that your sample meets the conditions for.
2. Get the prediction for that decision path.
3. Repeat this for all trees in the forest.
4. Take the average of all these predictions.

This should give you the same result as the predict() method. If it doesn't, there might be an issue with how you're determining which decision paths your sample meets the conditions for, or with how you're calculating the predictions for those decision paths.

Also, each tree in the forest is built on a different subset of the training data, and each split in each tree is based on a different subset of the features. So, it's possible for a sample to meet the conditions for a decision path in one tree but not in another, even if the decision paths look very similar.