# mAP scores on tensorboard (Tensorflow Object Detection API) are all 0 even though the loss value is low

I trained a faster-rcnn model on the tensorflow object detection API on a custom dataset. I found that the loss is ~2 after 3.5k steps. However, when I ran eval.py, the mAP scores are all almost 0 as shown below.

I do not understand why this is the case. However, when I look at the images at 3.5k steps, the model has captured some of the boxes as shown below

Can someone please explain why the mAP scores are close to zero, even though the model has learned to output quite a few boxes?

1. Try varying the Intersection over Union (IoU) threshold, e.g 0.2-0.5 and see if you get an increase in average precision. You would have to modify matching_iou_threshold parameter in object_detection/utils/object_detection_evaluation.py
2. Try different evaluator classes (the default one is EVAL_DEFAULT_METRIC = 'pascal_voc_detection_metrics'). If you are training on Open Image Dataset it makes sense to use open_images_V2_detection_metrics
eval_config: { num_examples: 20000 num_visualizations: 16 min_score_threshold: 0.2 # Note: The below line limits the evaluation process to 10 evaluations. # Remove the below line to evaluate indefinitely. max_evals: 1 }