0
$\begingroup$

I have an object detection model with my labels and images. I am trying to use the tensorflow ranking metric for MAP, https://www.tensorflow.org/ranking/api_docs/python/tfr/keras/metrics/MeanAveragePrecisionMetric. The metric is used when I compile the model but this is the result I get:

Epoch 2/220
92/92 [==============================] - 22s 243ms/step - loss: 0.0027 - mean_average_precision_metric: 0.0000e+00 - val_loss: 0.0019 - val_mean_average_precision_metric: 0.0000e+00
Epoch 3/220
92/92 [==============================] - 22s 245ms/step - loss: 0.0014 - mean_average_precision_metric: 0.0000e+00 - val_loss: 7.5579e-04 - val_mean_average_precision_metric: 0.0000e+00
Epoch 4/220
92/92 [==============================] - 23s 247ms/step - loss: 8.7288e-04 - mean_average_precision_metric: 0.0000e+00 - val_loss: 6.7357e-04 - val_mean_average_precision_metric: 0.0000e+00
Epoch 5/220
92/92 [==============================] - 23s 248ms/step - loss: 7.3901e-04 - mean_average_precision_metric: 0.0000e+00 - val_loss: 5.3464e-04 - val_mean_average_precision_metric: 0.0000e+00

My labels and images are all normalized as well according to my image dimensions.

train_images /= 255
val_images /= 255
test_images /= 255
train_targets /= TARGET_SIZE
val_targets /= TARGET_SIZE
test_targets /= TARGET_SIZE

model.compile(loss='mse', optimizer='adam', metrics=[tfr.keras.metrics.MeanAveragePrecisionMetric()])

Could the metric not be the right way of using it or maybe not meant for my data?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
0
$\begingroup$

I would look into whether your loss function is correct. Mean square error is a regression metric (and precision is a classification metric). Something like categorical cross entropy is probably more suited.

Eitherway as a sanity check I would you can always run a model for say 10 epochs. Then run predictions and calculate the precision manually (or with sklearns builtin method.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I have tried to change the loss function but there was no change in the result. However, I have tried to calculate the AP using the function with``` y_pred = np.array([106, 86, 115, 92]) y_truth = np.array([105, 85, 114, 91]) average_precision_score(y_pred, y_truth). However, I get an error called multiclass format is not supported```. This is with my predictions and ground truth labels. $\endgroup$
    – NevMthw
    Sep 7, 2022 at 8:14
  • $\begingroup$ @NevMthw this is because average_precision_score is expecting binary labels for classification. If you have 100 classes then you need to one-hot encode them to give something like this: y_pred =np.array([[1,0,0],[1,0,0],[1,0,0],[0,1,0],[0,0,1]]) y_truth = np.array([[1,0,0],[0,1,0],[1,0,0],[0,1,0],[0,0,1]]) average_precision_score(y_pred, y_truth) $\endgroup$
    – Adrian B
    Sep 14, 2022 at 10:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.