# LeNet-5 - combining feature maps in C3 layer

Famous LeNet-5 architecture looks like this:

The output of layer S2 has dimension: 10x10x6 - so basically an image with 6 convultions applied to it to derive features.

If each dimension was again submitted to 6 filters the resulting output would be of 10x10x36 however it is 10x10x16. Initially I stumble on it but finnaly I udnerstood that this is done be combining inputs from layer S2 and applying one kernel on it as it's explained in the article:

Layer C3 is a convolutional layer with 16 feature maps Each unit in each feature map is connected to several 5x5 neighborhoods at identical locations in a subset of S2s feature maps

.

The rationale behind the connection scheme in table I is the following The 1rst six C3 feature maps take inputs from every contiguous subsets of three feature maps in S2. The next six take input from every contiguous subset of four. The next three take input from some discontinuous subsets of four Finally the last one takes input from all S2 feature maps Layer C3 has 1,516 trainable parameters and 151,600 connections

and the roadmap of it is provided in the table:

What I am still not uderstand is how exactly should I combine them?

In previous layer I've just applied 6 kernels on 1 dimension, resulting in 6 dimensions what was understandable. Here I am a bit lost to be honest :(

• It is 16 kernels that take each take 6 channels. Any value the resulting 16 channels is a recombination of 5 * 5 * 16 values. It might be helpful to think of a filter over an RGB image (3 channels). Feb 1 '19 at 19:50
• Thanks @SvanBalen. Thinking like the rgb channels really helped. But as I undertstod this way table above. Is it that kernel 0 would be 5*5*3 but kernel 6 would be 5*5*4? Or all of them are as you said 5*5*6 but some of the channels that are not makred as "X" above are zero matrix? If so, how from having 16 kernels all of them have depth of 6 I achieve feature map of 5*5*16? Do they apply some compbination on 6 channels to receive depth o 1? Feb 2 '19 at 7:24
• I would think of it like this: each pixel in any of the 16 output layers is the result of a trainable operation applied over a spatial region of 5 by 5 on a 6 channel image. Yes some of these input channels are not considered for some of the output channels (I would think of that as rows of fixed zeroes in a tensor). Feb 2 '19 at 7:51
• Ok. and then do I just sum the results of those 6 channels? here I found this quote regarding convulving rgb picture: "If the image to be convolved has more than one channel, then the filter must has a depth equal to such number of channels. Convolution in this case is done by convolving each image channel with its corresponding channel in the filter. Finally, the sum of the results will be the output feature map.". Is it literally sum of corresponding cell in each of the channel? Feb 2 '19 at 7:54
• I think you are now asking how a convolution works in depth, perhaps this explanation helps: dsp.stackexchange.com/questions/8240/…. All you need to envision now is that that process work in a 3d space. Feb 2 '19 at 8:39

The filter $$0$$ of C3 is generated from the filter $$0,1,2$$ of S2. The kernel size is $$5x5$$. The number of parameter is $$5x5x3=75 + 1(bias) = 76$$. This applies to other filters as well. So overall, there are $$76 + 76 +...+101+151 = 1516$$ parameters.