Why are ROC curves better for imbalanced datasets?

Question

I have recently read this:

" AUC(Area Under Curve) is good for classification problems with a class imbalance. Suppose the task is to detect dementia from speech, and 99% of people don’t have dementia and only 1% do. Then you can submit a classifier that always outputs “no dementia”, and that would achieve 99% accuracy. It would seem like your 99% accurate classifier is pretty good, when in fact it is completely useless. Using AUC scoring, your classifier would score 0.5. "

Can someone please explain why does it reach 0.5? If 99% are negative and we output always "no", wouldn't that mean that the TruePositiveRate will be very high and the FalsePositiveRate very low, resulting in an Area Under Curve close to one?

Answer 1

The AUC reaches 0.5 because of this:

$Sensitivity = \frac{True Positives}{True Positives + False Negatives}$

$Specificity = \frac{True Negatives}{True Negatives + False Negatives}$

In your case:

$Sensitivity = \frac{99%}{99%+0%}=1$

$Specificity = \frac{0%}{0%+1%}=0$

$1-Specificity = 1$

Remember that the X axis is 1-Specificity.

The point $(1,1)$ is in the diagonal line that makes the AUC equal to 0.5

If you try this exercise with any other rule like "Saying 'yes' to dementia for all patients" you will get the point $(0,0)$ and joining these points you will have the diagonal line.

Answer 2

As per this answer, ROC curves are better than simple accuracy for imbalanced data sets, but they are still not good. It is better to use a precision-recall curve.