An activation function (say sigmoid) is necessary on the final fully connected layer. But why is an activation function applied on the convolution layer too? As I understand it, the activation function is needed to apply only once, on 1 final layer.
Not only for Convolutional Neural Networks (CNNs), also for DNNs (Deep Neural Networks) and RNNs (Recurrent Neural Networks), we use activation functions at every layer. Sigmoid (for binary classification), softmax (for multiclass classification) or some other types are usually used at the final output layer, each specific for the kind of labels that we have to compare with the predictions.
However, other neurons require activation functions as well, especially for nonlinearity purposes; most popular ones are ReLUs(Rectified Linear Units), Leaky ReLU, tanh,.. and so on. We nearly always use an activation function for every neuron in Deep Learning. For a detailed insight, have a look at:
Also specificly for Convolutional Nets:
Hope I could help, please do not hesitate to ask more. Good Luck!
Training a network is analogous to fitting a function to some scalar data. If the data is linear, fitting a linear function is appropriate and will work well.
In the case of deep learning, the data is rich and non-linear, so we apply non-linear activation functions to make the model more complex.
Another reason we use activation functions on intermediate layers is to keep the weight and output values close to 0 and "kind of" Gaussian, for optimization reasons.