Criteria for saving best model during training neural network?

Question

I am doing 4-class semantic segmentation with U-net using generalised dice loss as loss function.

General approach to save best model during training is to monitor validation loss at each epoch and save the model if val loss decreases than previous minimum.

But, I am interested in the "model which gives best the average dice score of 4 classes".

During training in my case, using validation loss as criteria doesn't lead to best avg dice score. So what should I consider the best model the one with less validation loss or highest validation dice score?

Below is my validation loss and avg dice score after each epoch. Out of this which epoch gives the best model?

epoch  1/10     validation loss: 0.95     avg dice score: 0.17
epoch  2/10     validation loss: 0.86     avg dice score: 0.23
epoch  3/10     validation loss: 0.77     avg dice score: 0.34
epoch  4/10     validation loss: 0.74     avg dice score: 0.40
epoch  5/10     validation loss: 0.71     avg dice score: 0.45
epoch  6/10     validation loss: 0.69     avg dice score: 0.34
epoch  7/10     validation loss: 0.79     avg dice score: 0.45
epoch  8/10     validation loss: 0.75     avg dice score: 0.51
epoch  9/10     validation loss: 0.76     avg dice score: 0.36
epoch 10/10     validation loss: 0.75     avg dice score: 0.38

If I go by val loss as criteria epoch-6 gives the best model, if i choose avg dice score as critieria epoch-8 gives the best model? how to choose?

Answer 1

The loss is mostly a measure that helps the model learn and is not looked at too much when deciding which model to select. A more business oriented measure is often used for this, e.g. accuracy. Since in this case you are mostly interested in the dice score it would make most sense to select the model from epoch 8.