Choose ROC/AUC vs. precision/recall curve?

Question

I am trying to get a clear understanding on various classification metrics, including knowing when to choose ROC/AUC as opposed to opting for the Precision/Recall curve.

I am reading Aurélien Géron's Hands-On Machine Learning with Scikit-Learn and TensorFlow book (page 92), where the following is stated:

Since the ROC curve is so similar to the precision/recall (or PR) curve, you may wonder how to decide which one to use. As a rule of thumb, you should prefer the PR curve whenever the positive class is rare or when you care more about the false positives than the false negatives.

The book demonstrates the ROC and PR curve for an imbalanced binary classification problem where target class is roughly 90% zero and 10% one. It shows the PR curve and the ROC curve where the above referenced ROC bias against imbalanced datasets is clearly reflected: ROC has an overly optimistic view of model performance.

Yet, I don't understand it thoroughly, why exactly is

• ROC overly optimistic for imbalanced binary classification problems, and
• PR curve favoring false positives over false negatives.

Generally speaking, I understand why precision and recall are useful for classification problems with an imbalance. For such problems, accuracy is highly biased. From precision we can infer the presence of false positives (the more FPs there are, the lower the precision) and similarly, from recall we can infer the presence of false negatives (the more FNs there are, the lower the recall).

However, when looking at the axes of the ROC curve, there true positive rate (TPR, recall) is plotted against false positive rate. The higher the x-axis (FPR) value, the more FPs there are. The lower the y-axis (TPR) value, the more FNs there are. This seems to be nearly analogous to the precision-recall curve though, where the lower the y-axis (precision) value, the more FPs there are and the lower the y-axis (recall) value, the more FNs there are. In other words, both ROC and PR curve seem to provide information on both FPs and FNs.

Answer 1

Although this is not a demonstration by an experimental exercise (we can actually try it out), you can get an intuitive understanding because, while PR-AUC uses Precision and Recall as indicators:

ROC-AUC uses Recall and FPR (False Positive Rate) which makes use of the TN (True Negatives) value which, for datasets with a high imbalance, is likely to be high and contributes to the overly optimistic ROC-AUC:

So, TN (a value less challenging for imbalanced datasets) is used in ROC-AUC, but not considered in PR-AUC.

Some references:

• Imbalanced Learning: Foundations, Algorithms, and Applications, 2013
• Learning from Imbalanced Data Sets, 2018
• A Survey of Predictive Modelling under Imbalanced Distributions, 2015
• https://machinelearningmastery.com/roc-curves-and-precision-recall-curves-for-imbalanced-classification/