I have a Neural Network with 4 classes (completely balanced) where the recall is the following for each class (the class with the highest score from the network is chosen)

$RE(C1) = 0.611650 $

$RE(C2) = 0.688172$

$RE(C3) = 0.580247$

$RE(C4) = 0.827160$

but doing a One-vs-Rest ROC-AUC I seem to be able to get a better performance thus the threshold for chosing each class might have to be tuned.

In the binary case it's simple, but how do we do it (if we even do it?) in the multi-class case, assuming the cost of FP and FN is equal and equal for all classes? I.e is there a simple way of writing a logic like "if $P(C1)>t1$ and $P(C2)<t2$ predict $C1$" where $ti$ is some threshold for class $i$?

Or is the best way simply just to chose the biggest score (softmax) from the network in the multi-class setting?

  • 1
    $\begingroup$ What happens when none of your classes meet the threshold for classification? That can’t happen in a binary problem, but it can in a multi-class problem. $\endgroup$
    – Dave
    Nov 24 at 13:50
  • $\begingroup$ How do you obtain the performance from a ROC curve? Do you mean the AUC score as performance? If so I don't see how you compare it to these recall values. $\endgroup$
    – Erwan
    Nov 24 at 18:49
  • $\begingroup$ @Dave agreed - it's a good question. I think I would flag it as "unknown" (e.g if a network trained for detection a mouse,cat and dog is used to predict on an image of an elephant, I would assume it be close to 0.25) $\endgroup$
    – CutePoison
    Nov 25 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.