What is the main goal of using an activation function in CNN?

I know the activation functions types and the purpose of each one. But here I am asking why to use them.


The idea of convolutional layers is that we need same weights to be applied to different regions of the input. It lets you identify same patterns that occur in different regions of the input. You can consider each convolution operation which is carried out by each window as a single neuron which just transforms a local region with a non-linear transformation. The linear and non-linear operations that are done by these so-called local neurons help the convolutional layers learn non-linear and complex features. Another aspect of this operation is that it reduces the number of weights significantly. The final interpretation is that if you don't use non-linear activations, you would have linear local features extracted from the input. Those linear features can easily be found with fewer number of parameters.

  • $\begingroup$ what do you mean by "Those linear features can easily be found with fewer number of parameters"? $\endgroup$ – N.IT May 6 '18 at 19:31
  • 1
    $\begingroup$ @N.IT take a look at here. If you don't use non-linear activation functions, it does not matter how many neurons and layers you add, they all can be replaced by a snigle one. $\endgroup$ – Media May 6 '18 at 19:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.