1
$\begingroup$

I am interested in finding the OOB score for random forest using sklearn, when it is used for a binary classification task, and there are unbalanced samples. What does the oob decision function mean in random forest, and how get class predictions from it?

I read RandomForestClassifier OOB scoring method but am still not clear. Does the oob decision function provide class probabilities, and if so, do I get the class predictions by taking whichever number is higher (e.g. by doing something like pred_train = np.argmax(forest.oob_decision_function_,axis=1))?

Since my classes are unbalanced, would it be correct to say I can't used sklearn's default OOB score here, and I should do the above to get some kind of F1 score from the OOB predictions, to get a better estimate of my random forest's error?

$\endgroup$
0
$\begingroup$

Every Tree gets its OOB sample.
So it might be possible that a data point is in the OOB sample of multiple Trees.

oob_decision_function_ calculates the aggregate predicted probability for each data points across Trees when that data point is in the OOB sample of that particular Tree.

The reason for putting above points is that OOB will give you the mean of probability but it will not tell you anything about the standard deviation of the probability across Trees.

Does the oob decision function provide class probabilities,

Yes

and if so, do I get the class predictions by taking whichever number is higher (e.g. by doing something like pred_train = np.argmax(forest.oob_decision_function_,axis=1))?

Yes

Since my classes are unbalanced, would it be correct to say I can't use sklearn's default OOB score here

OOB score is still the default score i.e. Accuracy. So, will not help for the Imbalanced class.

$\endgroup$
3
  • $\begingroup$ Thanks, that helps me a lot! =). Since OOB is calculated using accuracy, it would be possible to calculate OOB using another metric besides accuracy right? So basically I can do the following: 1) get class probabilities from OOB 2) get class predictions 3) calculate F1 score from such predictions 4) the above would get me the OOB score calculated using F1 right? $\endgroup$ – Jonathan Ng Feb 1 at 9:07
  • $\begingroup$ Yes for all 4 points. You may mark the Answer as accepted. Thanks. $\endgroup$ – 10xAI Feb 1 at 9:16
  • $\begingroup$ Ok! thanks that helps a lot! =) $\endgroup$ – Jonathan Ng Feb 1 at 10:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.