Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Lets say:- ML Model denied 1000 transactions FDR= FP/(FP+TP)
I randomly picked 400 accounts and checked if the model denied good user or bad user.
Good user denied- 110
Bad user denied - 290
How will I calculate False Discover rate of the model. Can it be based on the sample that I checked manually and then extrapolate ?
In your case, you can estimate the FDR by $$110 / 400 = 27.5\%$$
the approach of sampling some data and estimating the empirical FDR in there is valid, it's the same as estimating the mean of a population using a sample, it makes sense, and the more data you use the more confident you're going to be on your estimate, as long as you're sampling randomly.
400 seems a pretty decent number to estimate FDR, so your estimate is going to be pretty good.
