0
$\begingroup$

I need to calculate the accuracy but at a false positive rate of 1%. I am not sure if it is the normal accuracy that we can calculate with sklearn or I need a customized formula?

$\endgroup$
1
$\begingroup$

Calculate this by finding the threshold at which the false positive rate is 1%. Your model outputs a probability, not a category. You get a category by seeing on which side of some threshold, probably 0.5, the probability lies. Test some other thresholds. You can use the ROC curve for this.

Once you have that threshold, calculate the accuracy.

However, be cautious about accuracy (or any threshold-based scoring rule, such as F1 score) as a scoring rule.

https://stats.stackexchange.com/questions/312780/why-is-accuracy-not-the-best-measure-for-assessing-classification-models/312787#312787

https://stats.stackexchange.com/questions/464636/proper-scoring-rule-when-there-is-a-decision-to-make-e-g-spam-vs-ham-email

https://stats.stackexchange.com/questions/494023/proper-scoring-rule-in-optical-character-recognition

(Frank Harrell’s blog that gets linked is a good read, too.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.