Scheduler for activation layer parameter using Keras callback

Question

I want to train a keras model with a custom activation layer. The custom activation layer has one fixed non trainable parameter.

I want to change/set this non trainable parameter of all custom activation layers in model during training after few epochs.

How to achieve this using keras callback?

Answer 1

You will need to write a custom callback for this, that implements the on_epoch_end method. Roughly it should look something like this

class CustomCallback(keras.callbacks.Callback):

    def __init__(self, freq):
        super().__init__()
        self.freq = freq   # how often to change the parameter

    def on_epoch_end(self, epoch):

        if epoch % freq == 0 and epoch > 0:

            weights = self.model.get_weights()

            # here you change the weight you want, e.g. it is the 5th layer
            weights[4] = weights[4] / 10

            self.model.set_weights(weights)