# Activation Functions

What is the purpose of linear activation functions in keras, isn't the entire point of activation functions to introduce non-linearity?

• I'm not 100% sure, but the Activation ("linear") might be an historic artifact. There is / was also an activation parameter to a couple of functions. There it makes sense, because sometimes you don't want to apply an activation function. – Martin Thoma Nov 29 '18 at 7:03

A combination of multiple linear functions can help one model complex decision regions. Two or more linear functions can be combined to form a piece-wise approximation of a non linear function. Computation of non-linear activation function on a very deep network can be expensive (because of the exponential term involved). ReLu activation function which is a linear activation is used almost by every CNN model and is proved to give better results. $$f(x) = max(0,x-1) + max(0,x+1)$$ Consider the function above which is combination of two reLu's. This approximates the sigmoid function(try plotting the same) and is computationally cheap.