2
$\begingroup$

For the imbalanced datasets:

  1. Can we say the Precision-Recall curve is more informative, thus accurate, than ROC curve?
  2. Can we rely on F1-score to evaluate the skillfulness of the resulted model in this case?
$\endgroup$
  • $\begingroup$ Yes. Precision-Recall curve is absolutely reliable for imbalanced datasets when compared with ROC curve. F-1 score is so far one of the most reliable metric to measure the performance of a model. You are thinking in the right direction. $\endgroup$ – Adhira Deogade May 7 at 18:07
  • 1
    $\begingroup$ @AdhiraDeogade Thank you very much for your note. very valuable $\endgroup$ – Dave May 7 at 18:17
  • $\begingroup$ Sure. Keep asking. $\endgroup$ – Adhira Deogade May 7 at 18:18
  • $\begingroup$ @AdhiraDeogade Reliable with respect to what? It seems there is another measure apart from Precision-Recall/ROC/F1 that you are using to judge these. $\endgroup$ – JTH May 7 at 23:35
  • 1
    $\begingroup$ I just think it is reckless to say "measure A is better than measure B" when no evidence or benchmark is given. I think the value of one measure over another additionally depends on what the practitioner intends to do with the model, unless we are just talking about the intrinsic value of one model over another, then I believe proper scoring rules should be used. $\endgroup$ – JTH May 8 at 2:29
0
$\begingroup$

Precision-recall curve are argued to be more useful than ROC curve in "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets" by Saito and Rehmsmeier. They argue that ROC might lead to wrong visual interpretation of specificity.

F1-score equally balances precision and recall. In some domains it might be more useful to more heavily weight precision (F < 1) or recall (F > 1).

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.