I am trying to get a clear understanding on various classification metrics, including knowing when to choose ROC/AUC as opposed to opting for the Precision/Recall curve.
I am reading Aurélien Géron's Hands-On Machine Learning with Scikit-Learn and TensorFlow book (page 92), where the following is stated:
Since the ROC curve is so similar to the precision/recall (or PR) curve, you may wonder how to decide which one to use. As a rule of thumb, you should prefer the PR curve whenever the positive class is rare or when you care more about the false positives than the false negatives.
The book demonstrates the ROC and PR curve for an imbalanced binary classification problem where target class is roughly 90% zero and 10% one. It shows the PR curve and the ROC curve where the above referenced ROC bias against imbalanced datasets is clearly reflected: ROC has an overly optimistic view of model performance.
Yet, I don't understand it thoroughly, why exactly is
- ROC overly optimistic for imbalanced binary classification problems, and
- PR curve favoring false positives over false negatives.
Generally speaking, I understand why precision and recall are useful for classification problems with an imbalance. For such problems, accuracy is highly biased. From precision we can infer the presence of false positives (the more FPs there are, the lower the precision) and similarly, from recall we can infer the presence of false negatives (the more FNs there are, the lower the recall).
However, when looking at the axes of the ROC curve, there true positive rate (TPR, recall) is plotted against false positive rate. The higher the x-axis (FPR) value, the more FPs there are. The lower the y-axis (TPR) value, the more FNs there are. This seems to be nearly analogous to the precision-recall curve though, where the lower the y-axis (precision) value, the more FPs there are and the lower the y-axis (recall) value, the more FNs there are. In other words, both ROC and PR curve seem to provide information on both FPs and FNs.