1
$\begingroup$

Suppose I have 5 classes, denoted by 1, 2, 3, 4, and 5, and this is used in object detection.

When evaluating an object detection performance, suppose I have classes 1, 2, and 3 present, but classes 4 and 5 are not present in the targeted values.

Will each of classes 4 and 5 have average precision of 0 (due to its precision being zero as no true positives can be identified)? Or perhaps there are other considerations to take when this case occurs?

$\endgroup$

1 Answer 1

0
$\begingroup$

I'm not sure if there is a standard for this kind of case. I think it depends whether the system predicts any of these classes, i.e. whether there are any false positive cases:

  • if yes, then the precision must be zero indeed: out of all the instances predicted positive none was a true positive.
  • if no, I think it makes more sense to consider the precision as undefined (NaN), since no instance is predicted positive at all (that is, there is a division by zero in the calculation of precision).

This is a very borderline case, since it would be very questionable to use a test set which doesn't contain all the classes contained in the training set: such a test set cannot is not fit for the purpose of evaluation.

$\endgroup$
2
  • $\begingroup$ I see, that makes sense @Erwan . And yes, although the test case is questionable if this case occurs, should there be no false positive cases and thus the precision is NaN, then how should we calculate the mean average precision? Should the average precision of such class(es) be made 1 as no target class is present and no false positive on that class, or should it be 0 which means the mean average precision would be affected negatively? $\endgroup$ Aug 7, 2020 at 1:19
  • 1
    $\begingroup$ @user8152821 I think the mean average precision should be NaN as well in this case, reflecting the fact that there is no way to know in the data. $\endgroup$
    – Erwan
    Aug 7, 2020 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.