How are scores calculated for each class of binary classification

Question

The formula for Precision is TP / TP + FP, but how to apply it individually for each class of a binary classification problem,

For example here the precision, recall and f1 scores are calculated for class 0 and class 1 individually, I am not able to wrap my head around how these scores are calculated for each class individually.

Can someone please explain to me with this confusion matrix as an example?

Please explain in laymen's terms if possible.

Thank you

Answer 1

Your confusion matrix does not correspond to your classification report. Also the matrix that you show is not standard:

• the labels "True Positive" and "True negative" are confusing because these terms should only be used for the classification status (see below). They mean "true class is positive" and "true class is negative".
• It has the true classes as columns and the predicted classes as rows, whereas it's usually presented with true classes as rows and the predicted classes as columns.

The same as a regular confusion matrix:

 true classes
  |
  |
  v
      0    1    <---- predicted classes
  0   15   10
  1   15   60

For example there are 10 instances which have true class 0 but predicted class 1.

The first thing to define clearly is which class is considered as the positive class, because everything else depends on that. Here let's assume that class 1 is positive, 0 is negative. Now we can obtain the number for every classification status:

• An instances which has true class 1 and predicted class 1 is a true positive, meaning that it is predicted positive and its prediction is correct (same as true class). In the example there are 60 TP.
• An instances which has true class 0 and predicted class 0 is a true negative, meaning that it is predicted negative and the prediction is correct (same as true class). In the example there are 15 TN.
• An instances which has true class 0 and predicted class 1 is a false positive, meaning that it is predicted positive but the prediction is incorrect (different than true class). In the example there are 10 FP.
• An instances which has true class 1 and predicted class 0 is a false negative, meaning that it is predicted negative but the prediction is incorrect (different than true class). In the example there are 15 FN.

Once the above is clear it's straightforward to apply the formula, for instance for precision:

$$P=\frac{TP}{TP+FP}=\frac{60}{60+10}=0.86$$

Keep in mind that the obtained score is for class 1 as the positive class. In order to obtain precision for the other class, you need to define it as the positive class and redo the classification status.

Answer 2

First I will try to explain it in words - for me it always help to grasp the idea. So class precision supposed to measure how precise is your prediction given the class you predicted.

For example lets say you want to predict 'rainy' or 'not-rainy' for tomorrow. It might be that when your model predicts 'rainy' the probability of being correct is higher than when you predict not-rainy. That why you want to measure the precision separately.

In your example given that your model predicted the class 1 the probability that the true label is actually 1 according to the test set results is 2% which is in fact:

Precision =TP/(TP+FP)

TP = the model correctly classified the sample as 1

TP+FP = the model classified the sample as 1 (either correctly or incorrectly)

Answer 3

Notice the terminology that precision and recall both depend on "positive" predictions and actual "positives". Both of the classes in binary classification can be considered as "positive".

In the classification report that you shared, there are two classes: 0 and 1.

Case 1: We consider 1 as the positive class.

Here, predicted positives mean the number of data points which we have predicted as 1 and actual positives mean the number of data points which actually belong to class 1.

Case 2: We consider 0 as the positive class. Here, the predicted positives mean the number of data points which we have predicted as 0 and actual positives mean the number of data points which actually belong to class 0.

Observe that the confusion matrix of counts are different in both cases and therefore the percentages/probabilities are different as well.

In your given confusion matrix:

60 10

15 15

Let the first class be 1 and the second be 0. In it's given form, say we considered 1 as the positive class and 0 as the negative class,

precision = TP/(TP+FP) = 60/(60+10) = 0.856

recall = TP/(TP+FN) = 60/(60+15) = 0.8

Now, let's consider 0 as the positive class and 1 as the negative class.

Then 15 times, the data points were positive and predicted positive 60 times, the data points were negative and predicted negative. 10 times, the data points were positive but predicted negative 15 times, the data points were negative but predicted positive.

ie the confusion matrix after considering 0 as the positive class looks like

15 15

10 60

Here,

precision = TP/(TP+FP) = 15/(15+15) = 0.5

recall = TP(TP + FN) = 15/(15+10) = 0.6

which are different from the precision and recall we obtained in the first case.

The choice of positive class is only a matter of convention and should be decided by the Data Scientist according to the problem at hand.