AUC-ROC of a random classifier

Question

Why the area under the ROC Curve for a random classifier is equal to 0.5 and has diagonal shape? For me a random classifier would have 25% of TP,TN,FP,FN and therefore it would only be a single point on the ROC Curve.

Answer 1

Premise: The confusion matrix that you mention above is only correct if

1. there is as much Positive and Negative cases in the data
2. we use a random classifier that assigns Positive and Negative class both with probability 0.5

So you derive the confusion matrix for a very particular random classifier for a very particular dataset. This very particular random classifier is indeed, as you point out, only a point in the ROC plot. To obtain the whole ROC curve, we have to vary the probability with which we assign the positive class, from 0 to 1.

So in effects, the ROC curve is a graphical evaluation of the performance of infinitely many classifiers!

Each one of these random classifiers with a different probability will have a different expected confusion matrix.

---

Derivation: Here I derive the AUC of a random classifier on a dataset with an arbitrary class imbalance.

Assuming that there is a fraction $x$ of positive cases and a fraction $1-x$ of negative cases, and that our classifier consists of randomly assigning the positive class with probability $\rho$ and the negative class with probability $1-\rho$. The confusion matrix of a random classifier will have the following expected proportions $$TP = \rho x \\ FP = \rho (1-x) \\ FN = (1-\rho)x \\ TN = (1-\rho)(1-x)$$

Then we calculate the True Positive Rate (sensitivity) and False Positive Rate (1-specificity) of our random classifier

$$TPR=\frac{TP}{TP+FN}=\frac{\rho x}{\rho x + (1-\rho)x}=\rho\\ FPR=\frac{FP}{TN+FP}=\frac{\rho(1-x)}{\rho(1-x)+(1-\rho)(1-x)}=\rho$$

So as you can see TPR and TFR do not depend on the class proportion $x$, which means that the ROC-AUC will also be independent of $x$. Moreover TPR=FPR, meaning that the ROC-curve will look like a straight line from (0,0) to (1,1). Now we calculate the AUC as the integral between 0 and 1 of the area under TPR as a function of FPR as we vary the threshold $\rho$.

$$AUC=\int_0^1 \rho d\rho' = \frac{\rho'^2}{2}|_0^1 = 1/2$$

So the area under the ROC curve for a random classifier is 0.5 regardless of the class proportion.

Answer 2

A couple of definitions:

• True positive rate (TPR): Probability that a positive is labeled as positive.

• False positive rate (FPR): Probability that a negative is labeled as positive

ROC curves are plotted by varying the threshold of output score of the classifier (above which an instance is classified as positive, and below which, it is negative), computing and plotting the true positive rate (y-axis) and false positive rate (x-axis) for each threshold value.

A "random" classifier assigns a score sampled from the uniform distribution between 0 and 1, to each instance. If the threshold selected is 'x', then any instance having score above 'x' is positive. For each instance (irrespective of whether it is actually positive or negative), the probability of being labeled positive is 1-x. As x is varied between 1 and 0, both TPR and FPR vary between 0 and 1 (being equal at all points). Hence the line obtained is x=y (diagonal), and the area under this line can be computed as 0.5.

Answer 3

A simpler proof for Gino_JrDataScientist answer :-)

Assuming that your calssifier is a Bernoulli random variable with parameter $\rho$, and based on these definitions we will have: $$TPR=\frac{TP}{P}=\frac{\rho \, P}{P}=\rho\\ FPR=\frac{FP}{N}=\frac{\rho \, N}{N}=\rho \\ AUC=\int_0^1 \rho \, d\rho = \frac{1}{2}\rho^2 \big\rvert_0^1 = \frac{1}{2}$$