1
$\begingroup$

I am interested in evaluating a semantic segmentation network. I've seen lots of challenges such as PASCAL VOC use the mean average precision metric(mAP). I understand how this would work with an instance detection approach, however I am unsure how it works for networks which just provide a class label to each pixel. Should I treat each image as a single instance of each class?

Thanks

$\endgroup$
3
  • $\begingroup$ So, your question is how to calc mAP (mean average precision) for the semantic segmentation problem? $\endgroup$ Jan 8 '19 at 8:03
  • $\begingroup$ Yes, I am looking for a detailed explanation of this procedure. $\endgroup$
    – ben2789
    Jan 9 '19 at 2:52
  • $\begingroup$ Still looking for a detailed explanation. $\endgroup$
    – ben2789
    Jan 24 '19 at 21:03
0
$\begingroup$

Here is a great post which explains the evaluation methods for semantic segmentation: https://www.jeremyjordan.me/evaluating-image-segmentation-models/

It also describes the mean average precision at the end.

$\endgroup$
4
  • 1
    $\begingroup$ Maybe I am misinterpreting this post. It states that I should look at the iou of each mask, gt pair. This would be each class label for each image in the test set? I think this post explains how to implement mAP for object detection, not segmentation, is there any difference? $\endgroup$
    – ben2789
    Jan 9 '19 at 19:47
  • $\begingroup$ @ben2789 I think I agree with you. The link seems to describe segmentation for the first half of the article, and it is clear how to calculate a single value for both precision and recall. What is not clear to me, is how to calculate the curve, since that would require recalculating precision and recall while modifying some independent variable. What is that independent variable? IoU doesn't work, since both Precision and Recall increase for smaller IoU thresholds. For Object Detection, the independent variable is confidence, but my segmentation network doesn't output the confidence. $\endgroup$
    – craq
    Apr 8 '19 at 0:13
  • 1
    $\begingroup$ @craq is your models last layer a softmax activation function? if it is then you can get a pixel wise 'confidence' in each class, but that would consider each pixel an individual prediction, and this doesn't seem to agree with the metrics in the above link. $\endgroup$
    – ben2789
    Apr 9 '19 at 4:07
  • $\begingroup$ @ben2789 yes, I agree that pixelwise precision & recall is not what most people would understand when you quote mAP. I could make it work with softmax (or one of my models uses argmax, I could probably sort based on that too) but that doesn't seem right. $\endgroup$
    – craq
    Apr 9 '19 at 4:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.