0
$\begingroup$

I have a yolov3 model for object detection on 9 classes. What is the difference in computing metrics (such as mAP) on validation set and on a test set (unseen data)? What is usually done in literature and why?

$\endgroup$

1 Answer 1

1
$\begingroup$

I'm unsure about yolo however it may be possible that you want to tune some hyperparameters of the model.

As you may already know, optimizations based on the evaluation of training data make little sense. Therefore let's assume that you split your dataset into training and test data and train n models each with a different combination of hyperparameters. You could greedily select the best performing parameters based on the calculated metric on the test data and use those parameters to train the final model.

But which metric would you list for your resulting model? The corresponding metric evaluated on the test data? That would be very optimistic as those chosen parameters result in good metric results on that particular test data and because you optimized for that test data.

One solution is to split the data into train, validation and test data. Now you can optimize or choose the best hyperparameters by looking at the metric evaluated on the validation data. However, the final metric evaluation should be done on the so far unseen test data. Another solution is to use nested cross validation which is described here.

$\endgroup$
1
  • $\begingroup$ Thank you, very clear! I I have to figure out what YOLOv3 does (github.com/ultralytics/yolov3). By default it calucaltes mAP in validation, but I can calculated it also on test. $\endgroup$
    – Andrea
    Feb 18, 2022 at 7:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.