3
$\begingroup$

Why the area under the ROC Curve for a random classifier is equal to 0.5 and has diagonal shape? For me a random classifier would have 25% of TP,TN,FP,FN and therefore it would only be a single point on the ROC Curve.

$\endgroup$
  • $\begingroup$ You're describing a single point on an ROC curve, which is the particular case when you predict exactly 50% P and 50% N. You could also predict 80% P and 20% N, but still assign labels randomly. This is another point on the ROC curve. $\endgroup$ – Nuclear Wang May 22 '18 at 18:31
8
$\begingroup$

Premise: The confusion matrix that you mention above is only correct if

  1. there is as much Positive and Negative cases in the data
  2. we use a random classifier that assigns Positive and Negative class both with probability 0.5

So you derive the confusion matrix for a very particular random classifier for a very particular dataset. This very particular random classifier is indeed, as you point out, only a point in the ROC plot. To obtain the whole ROC curve, we have to vary the probability with which we assign the positive class, from 0 to 1.

So in effects, the ROC curve is a graphical evaluation of the performance of infinitely many classifiers!

Each one of these random classifiers with a different probability will have a different expected confusion matrix.


Derivation: Here I derive the AUC of a random classifier on a dataset with an arbitrary class imbalance.

Assuming that there is a fraction $x$ of positive cases and a fraction $1-x$ of negative cases, and that our classifier consists of randomly assigning the positive class with probability $\rho$ and the negative class with probability $1-\rho$. The confusion matrix of a random classifier will have the following expected proportions $$TP = \rho x \\ FP = \rho (1-x) \\ FN = (1-\rho)x \\ TN = (1-\rho)(1-x)$$

Then we calculate the True Positive Rate (sensitivity) and False Positive Rate (1-specificity) of our random classifier

$$TPR=\frac{TP}{TP+FN}=\frac{\rho x}{\rho x + (1-\rho)x}=\rho\\ FPR=\frac{FP}{TN+FP}=\frac{\rho(1-x)}{\rho(1-x)+(1-\rho)(1-x)}=\rho$$

So as you can see TPR and TFR do not depend on the class proportion $x$, which means that the ROC-AUC will also be independent of $x$. Moreover, TPR=FPR. Now we calculate the AUC as the integral between 0 and 1 of the area under TPR as a function of FPR as we vary the threshold $\rho$.

$$AUC=\int_0^1 \rho d\rho' = \frac{\rho'^2}{2}|_0^1 = 1/2$$

So the area under the ROC curve for a random classifier is 0.5 regardless of the class proportion.

|improve this answer|||||
$\endgroup$
2
$\begingroup$

A couple of definitions:

  • True positive rate (TPR): Probability that a positive is labeled as positive.

  • False positive rate (FPR): Probability that a negative is labeled as positive

ROC curves are plotted by varying the threshold of output score of the classifier (above which an instance is classified as positive, and below which, it is negative), computing and plotting the true positive rate (y-axis) and false positive rate (x-axis) for each threshold value.

A "random" classifier assigns a score sampled from the uniform distribution between 0 and 1, to each instance. If the threshold selected is 'x', then any instance having score above 'x' is positive. For each instance (irrespective of whether it is actually positive or negative), the probability of being labeled positive is 1-x. As x is varied between 1 and 0, both TPR and FPR vary between 0 and 1 (being equal at all points). Hence the line obtained is x=y (diagonal), and the area under this line can be computed as 0.5.

|improve this answer|||||
$\endgroup$
0
$\begingroup$

A simpler proof for Gino_JrDataScientist answer :-)

Assuming that your calssifier is a Bernoulli random variable with parameter $\rho$, and based on these definitions we will have: $$TPR=\frac{TP}{P}=\frac{\rho \, P}{P}=\rho\\ FPR=\frac{FP}{N}=\frac{\rho \, N}{N}=\rho \\ AUC=\int_0^1 \rho \, d\rho = \frac{1}{2}\rho^2 \big\rvert_0^1 = \frac{1}{2}$$

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.