0
$\begingroup$

I have trained a classifier outputting probabilities for each class. I want to tune the decision threshold in such a way that it accounts for different costs/gains assigned to false positives ($FP$), $FN$, $TP$, and $TN$.

Is there a streamlined way of doing this in scikit-learn?

$\endgroup$

1 Answer 1

0
$\begingroup$

The recent release of scikit-learn (May 2024, v1.5.0) introduces a new feature that facilitates cost-sensitive learning: TunedThresholdClassifierCV.

Its fixed-threshold sibling is FixedThresholdClassifier, allowing one to easily override the default 0.5 decision threshold. The release highlights motivate and outline these new additions.

They also include a new tutorial "Post-tuning the decision threshold for cost-sensitive learning", covering both constant and non-constant misclassification costs when tuning a classifier during cost-sensitive learning. A more basic tutorial on the new functionality is also available. The User Guide §3.3 has a relevant section as well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.