2
$\begingroup$

This was asked in viva of my ML course. I answered yes but could not precisely explain why. By 'better' I mean whether geometric interpretation gives more information than just the numeric score.

$\endgroup$
3
  • 4
    $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Dec 8, 2021 at 6:39
  • 2
    $\begingroup$ More specifically: you'll need to define what "better" means first, and as you do that you'll realize this question cannot be answered as it is. Please add some details and context indeed. $\endgroup$
    – Calimo
    Dec 8, 2021 at 8:04
  • 1
    $\begingroup$ As far as I think "better" here refers to the amount of information the geometric interpretation gives compared to the numeric value. I'm sorry I did not ask about context. $\endgroup$
    – Rohit
    Dec 8, 2021 at 11:42

3 Answers 3

2
$\begingroup$

Yes, the graph contains information that the AUC number alone does not have.

It is most interesting when comparing 2+ models that have very close AUC numbers. The graph can tell you that one model favours recall, and the other model favours precision. Or, if the lines are basically on top of the other, it tells you two models have the same performance at all thresholds.

The graph can also help you choose a threshold. So, if precision is more important, you are interested in the threshold where the line just starts to leave the left side. If precision is more important you are interested in the point where it just starts to leave the top side.

Flipping it around, looking at a graph requires a human expert. The AUC number allows you to compare thousands of models in the blink of an eye, and stay objective.

$\endgroup$
1
$\begingroup$

From the added "better" in the comments, I can interpret that to mean is there different information in the geometric representation vs the numeric representation. Ignoring if ROC is actually the right metric to use for the problem being solved. This information may not be relevant to the problem.

The numeric AUROC does not give insight into if the ROC curves cross. If the curves cross, there is a tradeoff between sensitivity and specificity. Choosing the higher AUROC does not allow you to know you made a decision on the tradeoff. This post explains it well and has a picture.

This brings up a point that many candidate models for a problem have similar ROC curves and selecting a model based on the AUROC or ROC may not be discriminating enough.

$\endgroup$
0
$\begingroup$

These is what I interpreted from your question:

Here is my answer: ROC-AUC Score is basically AUC (Area under the ROC Curve)..

IF you need more clarification:

ROC: will tell how well the classification model predicts given its threshold changes.

AUC: will tell you how the aggregrate measures across all the threshold. AUC will be 0 if all the predictions are wrong and 1 if all the predictions are right. Properties of AUC:

  1. AUC will tell you how well the predictions are ranked rather than its absolute value irrespective of scale.
  2. It will tell you quality of predictions irrespective of classification threshold (threshold invariant).

Use AUC for comparing the two classification models (e.g if logistic regression has a AUC = 0.8 & Random Forest with a AUC=0.9 go for Random Forest).

AUC has its limit:

  1. Scale-invariant is not always desirable. Specially when we want to calibrate our outputs.
  2. threshold invariant not desirable when we want to prioritize minimizing any of the rate (e.g prioritizing minimizing false positives in SPAM filter model)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.