2
$\begingroup$

Have looked at data on oob but would like to use it as a metric in a grid search on a Random Forest classifier (multiclass) but doesn't seem to be a recognised scorer for the scoring parameter. I do have OoB set to True in the classifier.
Currently using scoring ='accuracy' but would like to change to oob score.
Ideas or comments welcome

$\endgroup$
1

3 Answers 3

1
$\begingroup$

Check out the make_scorer function in sklearn: http://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html

You may need to code the OoB component yourself, I haven't ever used that metric for scoring before with Sklearn, so I don't know for sure.

Also, consider using random search rather than grid search, it is more practical when dealing with a large hyper parameter space, since it does not need to search the entire dimension of a parameters that have no impact.

$\endgroup$
2
  • $\begingroup$ Thanks for your response Jinglesting . I did try the make scorer - unsuccessfully but will now try to code the OOB metric . $\endgroup$
    – RDATA
    Aug 28, 2018 at 14:34
  • $\begingroup$ No problem RDATA, if this answer is correct for you, please don't forget to mark it as so! $\endgroup$ Aug 28, 2018 at 15:18
1
$\begingroup$

Sorry - I know this is on old thread but you can try calling model.oob_score_ after running your model fit. It should give the OOB accuracy

$\endgroup$
1
  • 1
    $\begingroup$ How will you do this with GridSearchCV is the question. $\endgroup$
    – Pandian Le
    Jan 4, 2021 at 7:54
0
$\begingroup$

I hope I'm phrasing this correctly.

You can call it as a attribute from the best estimator, i.e.

grid_model.best_estimator_.oob_score_

so you can't really use grid_model.oob_score_ because gridsearchcv does not have such an attribute; but you can call the best instance of the random forest model, then call the attribute.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.