3
$\begingroup$

The terms AUC and accuracy has been confusing for me.
In which circumstance AUC rate and accuracy rate become exactly equal?

$\endgroup$

3 Answers 3

5
$\begingroup$

AUC (or most often AUROC = "area under receiver operating characteristic") and accuracy are different measures, but used for same purpose - to objectively measure performance of a simple binary classifier.

The two measures can be equal at extreme values of 0 and 1 for perfect classifiers - or inverse perfect classifiers (you can just invert the output to get a perfect classifier). It might be possible for the values to be numerically equal at other times, but if so it would be a coincidence with no specific meaning.

Both these metrics can be used with a simple classifier that only outputs true or false values for class membership. However, the AUROC metric requires some kind of parameter you can vary in order to plot the ROC curve. Usually this is a threshold for classification, used against the class probability output of a classifier.

There are other possible metrics. For example, F1 score and cross entropy. The F1 score again has 1.0 for a perfect classifier and 0.0 for a bad classifier, but cross entropy scores lower for better classifiers - 0.0 for perfect and no upper limit for bad output. Again, the values of these might be equal to other metrics at some points, but if so it is not meaningful.

If you are comparing two classifiers for a particular task, then it is important to compare them using the same metric, on the same test data. The metric and test data you choose should relate to your original problem.

If you are reading other people's published results, and want to compare them, it is not really possible if one person has used AUC and the other accuracy.

$\endgroup$
0
$\begingroup$

It is not correct to say that equal accuracy and ROC AUC statistics "would be a coincidence with no specific meaning". If you are performing binary classification and construct a ROC curve based on predicted labels (0 or 1) as opposed to a continuously valued probability, then the area under the ROC curve will be the same as accuracy (to convince yourself of this, check out the visuals on one of the answers at https://stackoverflow.com/questions/31159157/different-result-with-roc-auc-score-and-auc). If you are using sklearn and use .predict() rather than .predict_proba(), you could get slightly different accuracy and ROC AUC values due to the sampling used in generating the ROC curve, but the "correct" approach is to use your estimator's .predict_proba() function, in which case the accuracies and ROC AUC's will not be the same (and similar values then would be pure coincidence).

$\endgroup$
1
  • $\begingroup$ This doesn't appear to be true. The linearly-interpolated ROC curve without probabilities as you describe has area equal to the mean of the recall and specificity at the nontrivial point; the accuracy is the "mediant" of those. These will be equal when (and only when) they or their denominators are equal. So, in the special case of balanced classification, it turns out to work (!), but otherwise it generally won't. But, see @saskra's recent answer regarding balanced accuracy scores. $\endgroup$
    – Ben Reiniger
    May 27, 2019 at 17:53
0
$\begingroup$

In the binary case, balanced accuracy is equal to the arithmetic mean of sensitivity (true positive rate) and specificity (true negative rate), or the area under the ROC curve with binary predictions rather than scores.

See: "Balanced accuracy score"

AUC = (tpr-fpr+1)/2 = (tpr+tnr)/2 = 1 – (fpr+fnr)/2

See: "From Precision, Recall and F-Factor to ROC, Informedness, Markedness & Correlation"

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.