0
$\begingroup$

Lets say:- ML Model denied 1000 transactions FDR= FP/(FP+TP)

I randomly picked 400 accounts and checked if the model denied good user or bad user.

Good user denied- 110

Bad user denied - 290

How will I calculate False Discover rate of the model. Can it be based on the sample that I checked manually and then extrapolate ?

$\endgroup$
0
$\begingroup$

In your case, you can estimate the FDR by $$110 / 400 = 27.5\%$$

the approach of sampling some data and estimating the empirical FDR in there is valid, it's the same as estimating the mean of a population using a sample, it makes sense, and the more data you use the more confident you're going to be on your estimate, as long as you're sampling randomly.

400 seems a pretty decent number to estimate FDR, so your estimate is going to be pretty good.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.