# Using LeakyRelu as activation function in CNN and best alpha for it

Since if we do not declare the activation function, the default will be set as linear for Conv2D layer. Is it true to write:

model.add(Conv2D(32, kernel_size=(3, 3),
input_shape=(380,380,1))


I mean now by the written lines, the activation function for Conv2D layer is set as LeakyRelu or not?

Further, I want to know what is the best alpha? I couldn't find any resources analyzing it.

The code,

model.add(Conv2D(32, kernel_size=(3, 3), input_shape=(380,380,1))

will definitely transform the outputs from the Conv2D using the LeakyReLU activation given parameter alpha ( negative slope of ReLU ).
In ReLU, we simply set the activation to 0 for negative values. This causes the dying ReLU problem which leds to overfitting. Hence, we return $$x \alpha$$ instead of 0, so that the unit does not become non-functional.
If we look at TensorFlow's tf.nn.leaky_relu method, we will find that the alpha is 0.2.
Whereas in Keras' layers.LeakyReLU class, you will find the alpha is 0.3.