# Keras functional api explanation of activation() layer?

With the keras functional api it is possible to write something like this:

x = Activation('relu')(x)
x = Dense(8, activation='softmax')(x)


My question is whether the Activation() function is a separate Layer (equivalent to Dense(128, activation='relu') or if not... why and when this notation is used?

As stated in the docs, the activation layer in keras is equivalent to a dense layer with the same activation passed as an argument.

This would be equivalent

x = Dense(64)(x)
x = Activation('relu')(x)


is equivalent to

x = Dense(8, activation='relu')(x)


As per your example if the activation layer is used as a layer, this will act as a transformation of the outputs of the previous layer.

As you can see in the model

inputs = Input(shape=(784,))
x = Dense(32, activation='tanh')(inputs)
x = Activation('relu')(x)
predictions = Dense(8, activation='softmax')(x)


The output of the first layer is the result of a densely connected layer with a tanh function. Then these outputs will each be transformed by a relu function. You can see how this is simply a transformation and does not introduce any new model parameters using

model.summary()


• In my case multiple Conv1D Layers are concatenated and then the code above is added: x = Activation('relu')(x) x = Dense(8, activation='softmax')(x) .... does this make any sense? Why should one perform Activation before Dense when there is no Dense Layer before the Activation part? – Predicted Life Oct 1 '18 at 0:06
• So x = Activation('relu')(x) is only a transformation? – Predicted Life Oct 1 '18 at 0:10
• Yes in this case you would simply be transforming the outputs of the previous layer. – JahKnows Oct 2 '18 at 2:06