model.add(Conv2D(32, kernel_size=(3, 3), input_shape=(380,380,1))
will definitely transform the outputs from the
Conv2D using the LeakyReLU activation given parameter alpha ( negative slope of ReLU ).
Further, I want to know what is the best alpha? I couldn't find any
resources analyzing it.
In ReLU, we simply set the activation to 0 for negative values. This causes the dying ReLU problem which leds to overfitting. Hence, we return $x \alpha$ instead of 0, so that the unit does not become non-functional.
If we look at TensorFlow's
tf.nn.leaky_relu method, we will find that the alpha is 0.2.
Whereas in Keras'
layers.LeakyReLU class, you will find the alpha is 0.3.
So you can clearly get an idea of what the parameter's value should be. It's basically a hyperparameter which you need to adjust through trial-error observations.