What is AUC - ROC Curve?

Question

AUC - ROC curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability.

Is Roc the same as AUC?

Answer 1

Related concepts, but not the same.

ROC-receiver operating curve

AUC area under the curve

Thank this post for explanation :

Abbreviations

• AUC = Area Under the Curve.
• AUROC = Area Under the Receiver Operating Characteristic curve.

AUC is used most of the time to mean AUROC, which is a bad practice since as Marc Claesen pointed out AUC is ambiguous (could be any curve) while AUROC is not.

---

Interpreting the AUROC

The AUROC has several equivalent interpretations:

• The expectation that a uniformly drawn random positive is ranked before a uniformly drawn random negative.
• The expected proportion of positives ranked before a uniformly drawn random negative.
• The expected true positive rate if the ranking is split just before a uniformly drawn random negative.
• The expected proportion of negatives ranked after a uniformly drawn random positive.
• The expected false positive rate if the ranking is split just after a uniformly drawn random positive.

Going further: How to derive the probabilistic interpretation of the AUROC?

---

Computing the AUROC

Assume we have a probabilistic, binary classifier such as logistic regression.

Before presenting the ROC curve (= Receiver Operating Characteristic curve), the concept of confusion matrix must be understood. When we make a binary prediction, there can be 4 types of outcomes:

• We predict 0 while the true class is actually 0: this is called a True Negative, i.e. we correctly predict that the class is negative (0). For example, an antivirus did not detect a harmless file as a virus .
• We predict 0 while the true class is actually 1: this is called a False Negative, i.e. we incorrectly predict that the class is negative (0). For example, an antivirus failed to detect a virus.
• We predict 1 while the true class is actually 0: this is called a False Positive, i.e. we incorrectly predict that the class is positive (1). For example, an antivirus considered a harmless file to be a virus.
• We predict 1 while the true class is actually 1: this is called a True Positive, i.e. we correctly predict that the class is positive (1). For example, an antivirus rightfully detected a virus.

To get the confusion matrix, we go over all the predictions made by the model, and count how many times each of those 4 types of outcomes occur:

In this example of a confusion matrix, among the 50 data points that are classified, 45 are correctly classified and the 5 are misclassified.

Since to compare two different models it is often more convenient to have a single metric rather than several ones, we compute two metrics from the confusion matrix, which we will later combine into one:

• True positive rate (TPR), aka. sensitivity, hit rate, and recall, which is defined as $ \frac{TP}{TP+FN}$. Intuitively this metric corresponds to the proportion of positive data points that are correctly considered as positive, with respect to all positive data points. In other words, the higher TPR, the fewer positive data points we will miss.
• False positive rate (FPR), aka. fall-out, which is defined as $ \frac{FP}{FP+TN}$. Intuitively this metric corresponds to the proportion of negative data points that are mistakenly considered as positive, with respect to all negative data points. In other words, the higher FPR, the more negative data points will be missclassified.

To combine the FPR and the TPR into one single metric, we first compute the two former metrics with many different threshold (for example $0.00; 0.01, 0.02, \dots, 1.00$) for the logistic regression, then plot them on a single graph, with the FPR values on the abscissa and the TPR values on the ordinate. The resulting curve is called ROC curve, and the metric we consider is the AUC of this curve, which we call AUROC.

The following figure shows the AUROC graphically:

In this figure, the blue area corresponds to the Area Under the curve of the Receiver Operating Characteristic (AUROC). The dashed line in the diagonal we present the ROC curve of a random predictor: it has an AUROC of 0.5. The random predictor is commonly used as a baseline to see whether the model is useful.

If you want to get some first-hand experience:

• Python: http://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
• MATLAB: http://www.mathworks.com/help/stats/perfcurve.html

Answer 2

No AUC and ROC are two different things.

AUC stands for 'Area under Curve'. ROC stands for 'Receiver Operating Characteristic curve'.

ROC is a metric which gives a single value for quantifying the performance of a classification model, given a threshold. E.g., you may choose 0.5 as a threshold for a cat vs dog classifier. If the threshold is not known then we plot a graph with threshold on the x-axis and the ROC metric on the y-axis.

The area under such a curve (AUC) is what is known as AUROC! AUC is not limited to just ROC. It can also be used with other metrics like precision-recall curves.

Answer 3

From this post:

This is an animation about the construction of the ROC Curve and so the AUC. One sees clearly that each point of the ROC Curve comes from a different threshold used to classify the output of a binary classifier. The threshold defines which samples are predicted as 1 and which ones as 0. True Positive and False Positive rates are then computed. At every threshold corresponds a point on the ROC Curve.

Note: The animation might be more relevant for people working on time-series, but could also help to understand the AUC construction in other cases as a Sample, the input of the binary classifier, could be anything (image, time-serie window, input features,...)