2
$\begingroup$

I've been using YOLO v2 for object detection and have been trying to use mAP for validating the model.

For training I use a confidence threshold of 0.25, however I am unsure as to whether to apply this threshold to filter out predictions BEFORE calculating mAP in validation? or whether the threshold should be ignored and all proposals should be evaluated?

$\endgroup$

2 Answers 2

1
$\begingroup$

In the end the solution is not to apply the threshold prior to the mAP calculation. After researching further it turns out by definition that mAP is (roughly speaking) a measure of precision recall trade off calculated using all possible thresholds. By applying the threshold first you are not accounting for the higher recall the model would obtain by choosing a threshold less than or equal to the one applied.

$\endgroup$
0
$\begingroup$

I would try both and see how big the difference is. It will depend on how clusttered your images are and how you otherwise threshold the mAP score. Will you for example ignore any bounding boxes that are tiny, compared to the image size?

Secondly, I would say it also depends on what you are validating. If you are interesting in the model, tweaking it during training and training further, I'd probably not ignore any proposals. If you are validating results for a real-world deployment, where you know that the thershold of 0.25 will be used, then I would probably just also validate without the thresholded proposals. That would provide a validation metric that might best respresent the expected performance when used in the wild.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.