2
$\begingroup$

I fit the random forest to my dataset with a binary target class. I reset the probabilistic cutoff to a much lower value rather than the default 0.5 according to the ROC curve. Then I can improve the sensitivity (recall) but meanwhile sacrificed the precision.

Just wanna confirm that the default 0.5 is not much meaningful and a practical probabilistic cutoff was often derived from ROC curve in practice. Am I on the right track on the application of random forest and other tree based models.

$\endgroup$
1
  • 1
    $\begingroup$ True, but note that the output is just the average of an ensemble of trees, and cannot be interpreted as a 'pure' probability. For that you need to for example combine probabilistic trees, or use Platt-scaling to recalibrate the output. More information can be found here $\endgroup$
    – Archie
    Jan 5 '17 at 15:02
2
$\begingroup$

Yes, you are exactly right. 0.5 is just a heuristic, ROC curve and precision-recall curve give a much better idea of what the cut off should be. You can then use predict_proba, extract the probabilities and do the classification based on the cut-off you have inferred from ROC curve and. precision-recall curve

$\endgroup$
0
$\begingroup$

Yes, 0.5 is the standard cut-off value. It always depends on the business problem what threshold probability you should use to classify the values as 0 or 1.

e.g. - If you are building a fraud model, a person with fraudulent probability of 0.3 and above may make sense to be marked as fraud. Or if you are building some similarity matrix, then a value less than 0.7 could be taken at 0.

So, ROCR curve is helpful to identify the exact cut-off percentage for the specific business problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.