I am trying to understand an article Backpropagation In Convolutional Neural Networks

But I can not wrap my head around that diagram: enter image description here

The first layer has 3 feature maps with dimensions 32x32. The second layer has 32 feature maps with dimensions 18x18. How is that even possible ? If a convolution with a kernel 5x5 applied for 32x32 input, the dimension of the output should be $(32-5+1)$ by $(32-5+1)$ = $28$ by $28$.

Also, if the first layer has only 3 feature maps, the second layer should have multiple of 3 feature maps, but 32 is not multiple of 3.

Also, why is the size of the third layer is 10x10 ? Should it be 9x9 instead ? The dimension of the previouse layer is 18x18, so 2x2 max pooling should reduce it to 9x9, not 10x10.

  • $\begingroup$ Your observation is correct and it seems that the author has mistyped some values.. $\endgroup$ – Aditya Feb 27 '18 at 6:46
  • $\begingroup$ Just checking, is there any mention of padding? Padding would make feature map reduction a bit less intuitive!! Oveal, you are right the size convention here seems rather confusing. For example, "3 feature maps with dimensions 32x32", do you mean a 32x32 color image then (32x32x3), and three is the channel! Check the excellent book by Ian Goodfellow book freely available online: deeplearningbook.org $\endgroup$ – TwinPenguins Feb 27 '18 at 8:05

Actually I guess you are making mistake about the second part. The point is that in CNNs, convolution operation is done over volume. Suppose the input image is in three channels and the next layer has 5 kernels, consequently the next layer will have five feature maps but the convolution operation consists of convolution over volume which has this property: each kernel will have its width and height, moreover, a depth. its depth is equal to the number of feature maps, here channels of the image, of the previous layer. Take a look at here.

  • $\begingroup$ Oh, I see. So the convolutional layer will always have as many output feature maps as it has kernels, right ? $\endgroup$ – koryakinp Feb 27 '18 at 22:30
  • $\begingroup$ @koryakinp yes, the number of feature maps of the next layer is equal to the kernel of the current convolution layer. $\endgroup$ – Media Feb 28 '18 at 6:11
  • $\begingroup$ Not to whine, but I'm I to understand that a clarification to a part of your question is accepted as answer, in favor of my answer that covers both questions? $\endgroup$ – S van Balen Feb 28 '18 at 10:06
  • $\begingroup$ @SvanBalen It's a bit difficult to read your comment, I didn't get it. $\endgroup$ – Media Feb 28 '18 at 10:57
  • $\begingroup$ @Media the comment just above this one you mean? Well basically I protest against your answer being the accepted one. It gave an answer to the second question in the question, but not the first, while mine gives the same answer (albeit less detailed, without the comment added later), but also an answer to the first question. Respectfully, I think that the accepted answer should be the most complete answer, even just for archive purposes. $\endgroup$ – S van Balen Feb 28 '18 at 13:00

It could be a case of padding in combination with convolution strides: if you would pad the first layer with 2 zeroes on either side and use a stride of 2, you would end up with an 18 * 18 * x. The 3 channels on the input are most probably RG&B, which are fairly commonly scaled up to 32 feature maps.

  • $\begingroup$ How 3 input channels can produce 32 feature maps ? As I understand, it can only produce multiple of 3 feature maps. $\endgroup$ – koryakinp Feb 27 '18 at 15:10
  • $\begingroup$ That is not necessary at all! The kernels will just receive a 5 * 5 * 3 signal. They will then each produce a scalar, which results in 32 layers by consequence of the architecture. $\endgroup$ – S van Balen Feb 27 '18 at 18:27
  • $\begingroup$ I think something which is missing in most explanations of CNNs and which might be the source of the confusion here, is how a scalar is produced from the 5 * 5 * 3 input, as you explain it. The convolution between the each one of the input channels and the kernels is fairly easy to understand, as a 2-D convolution between 2 2-D inputs produces a scalar. Nonetheless if you have 3 channels processed this way in parallel, it remains that you end up with 3 scalars, and I fail to find a convincing explanation anywhere on google about how those 3 scalars are combined together. $\endgroup$ – Whynote Aug 25 '20 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.