# How to use LeakyRelu as activation function in sequence DNN in keras?When it perfoms better than Relu?

How do you use LeakyRelu as an activation function in sequence DNN in keras? If I want to write something similar to:

 model = Sequential()


What is the solution? Put LeakyRelu similar to Relu?

Second question is: what are the best general setting for tuning the parameters of LeakyRelu? When is its performance significantly better than Relu?

You can use the LeakyRelu layer, as in the python class, instead of just specifying the string name like in your example. It works similarly to a normal layer.

Import the LeakyReLU and instantiate a model

from keras.layers import LeakyReLU
model = Sequential()

# here change your line to leave out an activation

# now add a ReLU layer explicitly:


Being able to simply write e.g. activation='relu' is made possible because of simple aliases that are created in the source code.

$$f(x) = alpha * x \:\text{for} \: x < 0,$$ $$f(x) = x \:\text{for}\: x >= 0.$$