Keras Callback example for saving a model after every epoch?

Question

Can someone please post a straightforward example of Keras using a callback to save a model after every epoch? I can find examples of saving weights, but I want to be able to save a completely functioning model after every training epoch.

Answer 1

Setting 'save_weights_only' to False in the Keras callback 'ModelCheckpoint' will save the full model; this example taken from the link above will save a full model every epoch, regardless of performance:

keras.callbacks.ModelCheckpoint(filepath, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)

Some more examples are found here, including saving only improved models and loading the saved models.

Answer 2

Make sure to include epoch variable in your filepath. Otherwise your saved model will be replaced after every epoch.

filepath = "saved-model-{epoch:02d}-{val_acc:.2f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=False, mode='max')

For more examples, check here.

Answer 3

I wrote my own ModelCheckpoint class as I have to call a special save_pretrained method:

class ModelCheckpoint(Callback):

    def __init__(self, freq, directory):
        super().__init__()
        self.freq = freq
        self.directory = directory

    def on_epoch_begin(self, epoch, logs=None):
        if self.freq > 0 and epoch % self.freq == 0:
            self.model.save_pretrained(Path(directory, str(epoch)))

    def on_train_end(self, logs=None):
        self.model.save_pretrained(directory)

It always saves the model every freq epochs and at the end of the training.