# the relationship between the number of filters/kernels and the number of feature maps

The above image is from "Deep Learning Tutorial" by Yann LeCun and Marc'Aurelio Ranzato (see pages 73 and 81).

I don't understand why 64 kernels (from the input to layer 1) produce 64 feature maps while 4096 kernels (layer 2 to layer 3) give 256 feature maps.

In the linked tutorial, each kernel is 2-dimensional, and applied to a single channel/feature map from its input. To construct a feature map, the output from $N_{input\_channels}$ convolutions are added together - this is important as it allows to build feature maps that have interactions between feature maps in the previous layer.

From the diagram, the first input layer has 1 channel (a greyscale image), so each kernel in layer 1 will generate a feature map. However, once you have 64 channels in layer 2, then to produce each feature map in layer 3 will require 64 kernels added together. If you want 256 feature maps in layer 3, and you expect all 64 inputs to affect each one, then you usually need 64 * 256 = 16384 kernels. The value 4096 is coming from some other aspect of the architecture not shown in the diagram, such as dividing the feature map into groups so that each output layer only processes a fraction of the input layers.

There are a few notation and presentation differences between presentations and tutorials on convolutional networks, depending on the source. This is one of them. Other sources may arrange the kernels into a 4D structure: $N_{input\_channels} \times N_{output\_channels} \times K_{width} \times K_{height}$ to make the relationship between kernels and input/output more explicit.

Another way to show the same thing is to treat the kernels as 3D, and ensure that the kernel depth $K_{depth}$ is the same as the number of input channels $N_{input\_channels}$ - this is same thing mathematically as summing up multiple separate convolutions, and dimensions might be $N_{output\_channels} \times K_{width} \times K_{height} \times K_{depth}$

The output of a convolution layer is computed as the following:

the depth (No of feature maps) is equal to the number of filters applied in this layer (because each added channel is a result of one filter's feature map)

the width ( the same for height) is computed according to the following equation

$$W=\frac{W−F+2P}{S} + 1$$ where $$F$$ is the receptive field (filter width), $$P$$ is the padding and $$S$$ is the stride.

For more details see the examples in the following link

CS231n Convolutional Neural Networks for Visual Recognition