0
$\begingroup$

I have two algorithms running on a piece of data, both of which perform differently.

One of them (call it A) consistently gets a positive predictive value of about 0.75-0.78. Looking at the AUC of the Receiver operating characteristic it has a score of about 0.82

The second algorithm (let's call it B) consistently get's a lower predictive value of about 0.72-0.75. However this gets a higher AUC of the Receiver operating characteristic value of about 0.85.

Does this definitively indicate an error of some kind as AUC is so tightly associated with the positive predictive value? Or is this entirely reasonable, subject to other factors?

$\endgroup$
0
$\begingroup$

AUC of ROC is not affected by PPV (precision), but by recall (true positive rate) and fall-out (false positive rate).

As per your results, model A gets a better PPV hence a lower recall and a lower ROC AUC. Inversely, model B gets a better recall hence a better ROC AUC and a lower PPV.

So this is entirely reasonable due to the definition of the two metrics.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.