I am trying to get a clear understanding on various classification metrics, including knowing when to choose ROC/AUC as opposed to opting for the Precision/Recall curve.

I am reading Aurélien Géron's Hands-On Machine Learning with Scikit-Learn and TensorFlow book (page 92), where the following is stated:

Since the ROC curve is so similar to the precision/recall (or PR) curve, you may wonder how to decide which one to use. As a rule of thumb, you should prefer the PR curve whenever the positive class is rare or when you care more about the false positives than the false negatives.

The book demonstrates the ROC and PR curve for an imbalanced binary classification problem where target class is roughly 90% zero and 10% one. It shows the PR curve and the ROC curve where the above referenced ROC bias against imbalanced datasets is clearly reflected: ROC has an overly optimistic view of model performance.

enter image description here

Yet, I don't understand it thoroughly, why exactly is

  • ROC overly optimistic for imbalanced binary classification problems, and
  • PR curve favoring false positives over false negatives.

Generally speaking, I understand why precision and recall are useful for classification problems with an imbalance. For such problems, accuracy is highly biased. From precision we can infer the presence of false positives (the more FPs there are, the lower the precision) and similarly, from recall we can infer the presence of false negatives (the more FNs there are, the lower the recall).

However, when looking at the axes of the ROC curve, there true positive rate (TPR, recall) is plotted against false positive rate. The higher the x-axis (FPR) value, the more FPs there are. The lower the y-axis (TPR) value, the more FNs there are. This seems to be nearly analogous to the precision-recall curve though, where the lower the y-axis (precision) value, the more FPs there are and the lower the y-axis (recall) value, the more FNs there are. In other words, both ROC and PR curve seem to provide information on both FPs and FNs.


1 Answer 1


Although this is not a demonstration by an experimental exercise (we can actually try it out), you can get an intuitive understanding because, while PR-AUC uses Precision and Recall as indicators:

enter image description here

ROC-AUC uses Recall and FPR (False Positive Rate) which makes use of the TN (True Negatives) value which, for datasets with a high imbalance, is likely to be high and contributes to the overly optimistic ROC-AUC: enter image description here

So, TN (a value less challenging for imbalanced datasets) is used in ROC-AUC, but not considered in PR-AUC.

Some references:


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.