0
$\begingroup$

I'm training a CNN and using:

model_checkpoint = ModelCheckpoint(os.path.join(output_artifacts,'weights.h5'), monitor='val_acc', save_best_only=True)

I trained the network for 70 epochs, but the validation accuracy flattened on good value (90%) after 20 epochs remaining pretty constant.

enter image description here

enter image description here

I would say that after epoch 20 (more or less 20:00 in the graph) the network is overfitting. My question is, does save_best_only takes this into account or just save the weights at epochs 1:00 am which is the overall best (but overfitted)? Should I use earlystopping to stop the training or is not necessary? I don't want to stop the training too early..

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

It really only tracks the value of the metric you selected, there is no tolerance option. In the relevant documentation, the definition is given:

save_best_only: if save_best_only=True, the latest best model according to the quantity monitored will not be overwritten.

In your case, combing the callback with the EarlyStopping callback would be the best option. This callback does offer some kind of tolerance, i.e. it requires the improvement of the metric to be at least as good as min_delta to be considered an improvement.

You could combine both callbacks with something like this:

from keras.callbacks.callbacks import EarlyStopping

early_stop = EarlyStopping(monitor='val_loss', min_delta=0.05, patience=3, verbose=1, mode='auto')
checkpoints = ModelCheckpoint('model_weights.h5', monitor='val_acc', save_best_only=True)

my_callbacks = [early_stop, checkpoints]
...
model.fit(..., callbacks=my_callbacks)
Improve this answer
$\endgroup$
2
  • $\begingroup$ Is my network overfitting in your opion even if the validation loss stays prettty much constant in the long run ? $\endgroup$
    – rok
    Apr 2, 2020 at 9:45
  • 1
    $\begingroup$ Based on your plots, not really... Perhaps you have some form of regularisation that prevents overfitting (dropout is an example). You could plot train_acc with val_acc to see if they diverge at all after some time - then the model is overfitting. $\endgroup$
    – n1k31t4
    Apr 2, 2020 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.