Fitting probabilities in scikit-learn RandomForestClassifier

Question

This is a newbie questions, so please bear with me. Given this random forest model:

from sklearn.ensemble import RandomForestClassifier

X = [ [2,1,1,1], [2,0,2,1], [3,1,1,1] ]
y = [ 0, 1, 2 ]

regr = RandomForestClassifier(n_estimators=200, max_depth=5)
regr.fit(X, y)

X_test = [ [3, 1, 1, 1] ]    
y_result = regr.predict_proba(X_test)
print('y_result:' , y_result )

The result is:

y_result: [[0.26 0.05 0.69]]

I understand that these are the probabilities of the first, second and third value, or 0 = 26%, 1 = 5% and 2 = 69%.

Question: if the test set is [3, 1, 1, 1] and it fits to the value 2, why do I get 69% probability of 2 instead of 100%?

Answer 1

Well, based on how this algorithm works, here is why : You want to build 200 trees, with a max depth of 5 levels. Each tree is a decision tree, train with a subset of your data. To answer your question, it doesn't fit to the value 2 because that's not how it works. What's happening is that 52 trees predicted the value 0, 10 predicted the value 1, and 138 predicted the value 2. So most of the trees predicted the value 2 You asked for the probability (using predict_proba(X_test)). If you want only one value, don't use predict_proba but predict(X_test). But it will only be the reflect of the probabilities you got.