3
$\begingroup$

I am trying to understand CNN especially the maths and working mechanism using Matlab as the coding language. I have few confusion regarding the concept and the associated programming and will be immensely grateful for an intuitive answer.

Below is the structure of my CNN for 5 classes. I could calculate only the output structure of the first Conv layer and stuck on determining the number of parameters i.e., number of neurons?

The output for the first convolution layer that I could calculate: In the first layer an input of size [50 50 2] is convolved with a set of M_1 5-dimensional filters applied over all the input channels. The first 2 D convolutional layer is composed of M_1 = 20 filters of size [5x5x 1] having the step size (stride) for traversing the input vertically and horizontally as 1 creating a feature map of size {(h-f_h+1) x (w - f_w +1)x 1x M_1} = (50-5+1)x(50-5+1)x20 = [46x46x 20] So we have 20 channels.

$\endgroup$
1
$\begingroup$

For a CNN layer with input of dimensions h * w * d, kernel size k * k and number of kernel filters as f, we have the number of parameters as k * k * d * f, if we ignore the biases. If use biases then the number of parameters becomes (k * k * d + 1) * f

For e.g., the 1st conv layer has 5 * 5 * 2 * 20 parameters if we are ignoring the biases. With bias, the number of parameters would become (5 * 5 * 2 + 1) * 20.

Note that the number of parameters does not depend on the stride, padding, pooling, dropout nor on the spatial dimensions of the input or output!

To find the total number of parameters in the network one needs to add the parameters of all the individual layers in the model.

| improve this answer | |
$\endgroup$
  • $\begingroup$ (1) If your input is of d=2 then indeed the filter dimension should have been written [5 x 5 x 2]. In that case my calculation would also change. (My ans has been edited) (2) Yes, you need to add the number of parameters of all the layers to find the total number of parameters of the neural network. For the 2nd layer, the number of parameters becomes 5 * 5 * 20 * 40 as d=20 for the input to the 2nd conv layer whereas it there are 40 filters in the 2nd conv layer. $\endgroup$ – user1825567 Dec 27 '19 at 10:28
  • $\begingroup$ Thank you for the clarification. If I understood you clearly-- we do not use the dropout and the maxpooling in the calculation of the number of parameters, is that right? It will be useful if you could include the answer to comment (2) in your actual answer for better understanding it. Also, I could not understand how you are getting 40 and the last line "whereas it there are 40 filters in the 2nd conv layer." I really found your answer helpful. $\endgroup$ – Sm1 Dec 27 '19 at 20:10
  • $\begingroup$ Yes, you got it right that pooling and dropout does not affect the number of parameters. I got 40 from your code where you define your 2nd conv to have 2*numF number of filters where numF=20. I am glad you found my answer helpful. Please upvote my answer also :) $\endgroup$ – user1825567 Dec 28 '19 at 6:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.