9
$\begingroup$

I am trying to understand an article Backpropagation In Convolutional Neural Networks

But I can not wrap my head around that diagram: enter image description here

The first layer has 3 feature maps with dimensions 32x32. The second layer has 32 feature maps with dimensions 18x18. How is that even possible ? If a convolution with a kernel 5x5 applied for 32x32 input, the dimension of the output should be $(32-5+1)$ by $(32-5+1)$ = $28$ by $28$.

Also, if the first layer has only 3 feature maps, the second layer should have multiple of 3 feature maps, but 32 is not multiple of 3.

Also, why is the size of the third layer is 10x10 ? Should it be 9x9 instead ? The dimension of the previouse layer is 18x18, so 2x2 max pooling should reduce it to 9x9, not 10x10.

Improve this question
$\endgroup$
2
  • $\begingroup$ Your observation is correct and it seems that the author has mistyped some values.. $\endgroup$
    – Aditya
    Commented Feb 27, 2018 at 6:46
  • $\begingroup$ Just checking, is there any mention of padding? Padding would make feature map reduction a bit less intuitive!! Oveal, you are right the size convention here seems rather confusing. For example, "3 feature maps with dimensions 32x32", do you mean a 32x32 color image then (32x32x3), and three is the channel! Check the excellent book by Ian Goodfellow book freely available online: deeplearningbook.org $\endgroup$
    – TwinPenguins
    Commented Feb 27, 2018 at 8:05

3 Answers 3

Reset to default
4
$\begingroup$

Actually I guess you are making mistake about the second part. The point is that in CNNs, convolution operation is done over volume. Suppose the input image is in three channels and the next layer has 5 kernels, consequently the next layer will have five feature maps but the convolution operation consists of convolution over volume which has this property: each kernel will have its width and height, moreover, a depth. its depth is equal to the number of feature maps, here channels of the image, of the previous layer. Take a look at here.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Oh, I see. So the convolutional layer will always have as many output feature maps as it has kernels, right ? $\endgroup$
    – koryakinp
    Commented Feb 27, 2018 at 22:30
  • $\begingroup$ @koryakinp yes, the number of feature maps of the next layer is equal to the kernel of the current convolution layer. $\endgroup$
    – Green Falcon
    Commented Feb 28, 2018 at 6:11
  • $\begingroup$ Not to whine, but I'm I to understand that a clarification to a part of your question is accepted as answer, in favor of my answer that covers both questions? $\endgroup$
    – S van Balen
    Commented Feb 28, 2018 at 10:06
  • $\begingroup$ @SvanBalen It's a bit difficult to read your comment, I didn't get it. $\endgroup$
    – Green Falcon
    Commented Feb 28, 2018 at 10:57
  • $\begingroup$ @SvanBalen I have no idea :) but I can do something else +30, I hope it satisfies you to answer others questions in order to help the others as well :) $\endgroup$
    – Green Falcon
    Commented Feb 28, 2018 at 13:07
1
$\begingroup$

It could be a case of padding in combination with convolution strides: if you would pad the first layer with 2 zeroes on either side and use a stride of 2, you would end up with an 18 * 18 * x. The 3 channels on the input are most probably RG&B, which are fairly commonly scaled up to 32 feature maps.

Improve this answer
$\endgroup$
3
  • $\begingroup$ How 3 input channels can produce 32 feature maps ? As I understand, it can only produce multiple of 3 feature maps. $\endgroup$
    – koryakinp
    Commented Feb 27, 2018 at 15:10
  • $\begingroup$ That is not necessary at all! The kernels will just receive a 5 * 5 * 3 signal. They will then each produce a scalar, which results in 32 layers by consequence of the architecture. $\endgroup$
    – S van Balen
    Commented Feb 27, 2018 at 18:27
  • $\begingroup$ I think something which is missing in most explanations of CNNs and which might be the source of the confusion here, is how a scalar is produced from the 5 * 5 * 3 input, as you explain it. The convolution between the each one of the input channels and the kernels is fairly easy to understand, as a 2-D convolution between 2 2-D inputs produces a scalar. Nonetheless if you have 3 channels processed this way in parallel, it remains that you end up with 3 scalars, and I fail to find a convincing explanation anywhere on google about how those 3 scalars are combined together. $\endgroup$
    – Whynote
    Commented Aug 25, 2020 at 13:55
0
$\begingroup$

the reason you're confused is because some parameters was ommitted in the paper. Assume from first layer to second layer, with the stride=2 and pad=4 and 32 kernels, 18=(32+2*4-5)/2-1

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.