1
$\begingroup$

I'm trying to understand precision and recall with an intuitive example, but my calculation doesn't seem right.

For example, there are 8 red balls and 2 blue ones. I'm stupid and just predict all of them are red. The accuracy would be 0.8 and looks good. But it doesn't reflect that it's just a lucky, stupid guess, and I missclassify all of the blue balls. If I'm not mistaken, we have 8 true positives (TP = 8), 2 false positives (FP = 2), and neither true nor false negatives (TN = 0, FN = 0) in this case. Then isn't precision = TP/(TP+FP) = 0.8 and recall = TP/(TP+FN) = 1? Don't they still look good?

$\endgroup$
1
$\begingroup$

Your calculation is correct, but you forgot to ask yourself one question: why should we consider "red" as the positive class? Precision and recall can be calculated for every class (i.e. considering the current class as positive), as opposed to accuracy.

So if we take "blue" as positive we get:

  • precision = NaN (because there's 0 predicted positive)
  • recall = 0 / 2 = 0

Usually after that one would calculate micro- and/or macro-precision/recall:

  • macro-precision = NaN because the precision is NaN for blue
  • micro-precision = totalTP / totalTP+totalFP = 8/10
  • macro-recall = ( 1 + 0 ) / 2 = 0.5
  • micro-recall = totalTP / totalTP+totalFN = 8/10

By definition, micro-performance favours the majority class so it's still high. However macro-performance would clearly show the problem that accuracy cannot show.

$\endgroup$
2
  • $\begingroup$ You are right! Thank you very much! I'm so used to sensitivity and specificity that I expected one of them to automatically go down. $\endgroup$ – Paw in Data Oct 9 '20 at 18:58
  • 1
    $\begingroup$ Indeed, in practice we consider as positive the class of interest, which is normally the minority class (e.g. sick patients, faulty engine etc) and not the majority ("normal") one. $\endgroup$ – desertnaut Oct 12 '20 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.