1
$\begingroup$

Have looked at data on oob but would like to use it as a metric in a grid search on a Random Forest classifier (multiclass) but doesn't seem to be a recognised scorer for the scoring parameter. I do have OoB set to True in the classifier.
Currently using scoring ='accuracy' but would like to change to oob score.
Ideas or comments welcome

| improve this question | |
$\endgroup$
1
$\begingroup$

Check out the make_scorer function in sklearn: http://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html

You may need to code the OoB component yourself, I haven't ever used that metric for scoring before with Sklearn, so I don't know for sure.

Also, consider using random search rather than grid search, it is more practical when dealing with a large hyper parameter space, since it does not need to search the entire dimension of a parameters that have no impact.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your response Jinglesting . I did try the make scorer - unsuccessfully but will now try to code the OOB metric . $\endgroup$ – RDATA Aug 28 '18 at 14:34
  • $\begingroup$ No problem RDATA, if this answer is correct for you, please don't forget to mark it as so! $\endgroup$ – Jinglesting Aug 28 '18 at 15:18
1
$\begingroup$

Sorry - I know this is on old thread but you can try calling model.oob_score_ after running your model fit. It should give the OOB accuracy

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.