Evaluation Metrics for Multiclass

Question

How to obtain the Accuracy, Detection_Rate, False_Positive_Rate and False_Negative_Rate for each class ?

For example, all these metrics in class_1, class_2, class_3 etc.

Answer 1

Here is an example of a code I used to compute the accuracy of a VGG classifier, it is done with Pytorch :

# VGGClassifier is my Model being trained
# testloader is my dataset for testing
print('Now testing...')
res = 0  # This variable counts the number of good answer
for n, (Xtest, Ytest) in enumerate(testloader):  # Batchsize of testloader should probably be 1
    Xtest, Ytest = Xtest.to(device), Ytest.to(device)  # Put the tensor on CPU/GPU
    Y_pred = VGGClassifier(Xtest)  # Compute the output of the model
    if torch.argmax(Y_pred) == Ytest:
        res += 1  # If predicted output = groundtruth output, then we add 1 to the counter
acc = res / (len(testloader))  # divide the number of good answer by the total number
acctab.append(acc)  # This list stores the accuracy values during training
print("acc : ", acc)

This code is not very hard so take your time to understand it and you can then adapt it to compute Detection_Rate, False_Positive_Rate and False_Negative_Rate.

For example to compute Detection_Rate of class1, you use the same code as accuracy, but instead of checking the output for each value, you only check the output for values where Ytest = 1 (they belong to the first class).

Not sure if my explaination is clear enough, I can give some more code snippets if you struggle with some metrics.

I'm pretty sure TF and Pytorch both have the basic metrics already coded, so make sure to have a look on their library of metrics :

Pytorch metrics

TF