3
$\begingroup$

According to this article,

The ROC curve for a perfect model would go straight up the TPR axis on the left and then across the FPR axis at the top. Since the plot area for the curve measures 1x1, the area under this perfect curve would be 1.0

Is there a more formal reasoning that explains this statement? My guess is it has something to do with summing the TPR over all possible FPR, and a perfect model should have the highest possible TPR (i.e. $1$) over all possible FPR.

Improve this question
$\endgroup$
9
  • 4
    $\begingroup$ I find it a bit troubling to suggest that $ROCAUC=1$ and a perfect model are synonymous. $ROCAUC$ has a theoretical maximum value of one, yes, and a model that always predicts the correct category does have $ROCAUC=1$, but a model with $ROCAUC=1$ need not have all perfect predictions according to the predictions from the sklearn predict method or a perfect score in terms of log loss or Brier score. (Perhaps I’ll demonstrate in an answer later.) $\endgroup$
    – Dave
    Commented Jan 6 at 22:39
  • $\begingroup$ TPR = tournament performance rating. Not. What is it then? True positive rate? $\endgroup$
    – Peter Mortensen
    Commented Jan 7 at 22:00
  • $\begingroup$ AUC = area under curve $\endgroup$
    – Peter Mortensen
    Commented Jan 7 at 22:02
  • $\begingroup$ FPR = false positive rate? $\endgroup$
    – Peter Mortensen
    Commented Jan 7 at 22:03
  • $\begingroup$ FP = false positive? $\endgroup$
    – Peter Mortensen
    Commented Jan 7 at 22:06

2 Answers 2

Reset to default
3
$\begingroup$

The difficulty with ROC curves is to understand what happens when the threshold varies. There is no summing, the curve only depends on how many instances have TP/FP/TN/FN status for every threshold.

A perfect model separates the two classes perfectly, this is why it's impossible to have both FP and FN cases:

  • if there are FP cases, then the threshold is too low, so there are no FN cases since the two classes are perfectly separated. The FPR is higher than 0 but the TPR is exactly 1.
  • Conversely, if there are FN cases then the threshold is too high and therefore are no FP cases for the same reason. Here the TPR is lower than 1 but the FPR is exactly 0.

Example:

prediction value   gold status
1.0                P
0.95               P
0.92               P
0.87               P
0.82               N
0.82               N
0.82               N
0.73               N
0.67               N
0.52               N

In this example, the two classes are perfectly separated, i.e. all the Ps are together at the top and all the Ns are together at the bottom.

  • if the threshold is higher than 0.87, then we would have FN cases but no FP case at all. This corresponds to a vertical line from (0,0) to (0,1) on the ROC curve, since FPR is 0.
  • if the threshold is lower than 0.82, then we would have FP cases but no FN at all. This corresponds to a horizontal line from (0,1) to (1,1) since TPR is 1.
  • if the threshold is between .82 and .87, then TPR=1 and FPR=0, this is the top left corner on the curve.
Improve this answer
$\endgroup$
2
$\begingroup$

An explanation I find more insightful is by noting the probabilistic interpretation of AUC. For any random pair of positive and negative instance, AUC is the probability that the model gives a higher score to the positive instance than the negative instance.

Given that it's a probability, obviously the highest possible value is 1.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.