3
$\begingroup$

Famous LeNet-5 architecture looks like this: enter image description here

The output of layer S2 has dimension: 10x10x6 - so basically an image with 6 convultions applied to it to derive features.

If each dimension was again submitted to 6 filters the resulting output would be of 10x10x36 however it is 10x10x16. Initially I stumble on it but finnaly I udnerstood that this is done be combining inputs from layer S2 and applying one kernel on it as it's explained in the article:

Layer C3 is a convolutional layer with 16 feature maps Each unit in each feature map is connected to several 5x5 neighborhoods at identical locations in a subset of S2s feature maps

.

The rationale behind the connection scheme in table I is the following The 1rst six C3 feature maps take inputs from every contiguous subsets of three feature maps in S2. The next six take input from every contiguous subset of four. The next three take input from some discontinuous subsets of four Finally the last one takes input from all S2 feature maps Layer C3 has 1,516 trainable parameters and 151,600 connections

and the roadmap of it is provided in the table: enter image description here

What I am still not uderstand is how exactly should I combine them?

In previous layer I've just applied 6 kernels on 1 dimension, resulting in 6 dimensions what was understandable. Here I am a bit lost to be honest :(

Please help. Mateusz

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ It is 16 kernels that take each take 6 channels. Any value the resulting 16 channels is a recombination of 5 * 5 * 16 values. It might be helpful to think of a filter over an RGB image (3 channels). $\endgroup$
    – S van Balen
    Feb 1, 2019 at 19:50
  • $\begingroup$ Thanks @SvanBalen. Thinking like the rgb channels really helped. But as I undertstod this way table above. Is it that kernel 0 would be 5*5*3 but kernel 6 would be 5*5*4? Or all of them are as you said 5*5*6 but some of the channels that are not makred as "X" above are zero matrix? If so, how from having 16 kernels all of them have depth of 6 I achieve feature map of 5*5*16? Do they apply some compbination on 6 channels to receive depth o 1? $\endgroup$
    – Mateusz Konopelski
    Feb 2, 2019 at 7:24
  • 1
    $\begingroup$ I would think of it like this: each pixel in any of the 16 output layers is the result of a trainable operation applied over a spatial region of 5 by 5 on a 6 channel image. Yes some of these input channels are not considered for some of the output channels (I would think of that as rows of fixed zeroes in a tensor). $\endgroup$
    – S van Balen
    Feb 2, 2019 at 7:51
  • $\begingroup$ Ok. and then do I just sum the results of those 6 channels? here I found this quote regarding convulving rgb picture: "If the image to be convolved has more than one channel, then the filter must has a depth equal to such number of channels. Convolution in this case is done by convolving each image channel with its corresponding channel in the filter. Finally, the sum of the results will be the output feature map.". Is it literally sum of corresponding cell in each of the channel? $\endgroup$
    – Mateusz Konopelski
    Feb 2, 2019 at 7:54
  • $\begingroup$ I think you are now asking how a convolution works in depth, perhaps this explanation helps: dsp.stackexchange.com/questions/8240/…. All you need to envision now is that that process work in a 3d space. $\endgroup$
    – S van Balen
    Feb 2, 2019 at 8:39

2 Answers 2

Reset to default
0
$\begingroup$

See this table, each row of which has 16 numbers. I don't know how to align them, so I posted a picture as well.

FILTER 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
WEIGHTS 75 75 75 75 75 75 100 100 100 100 100 100 100 100 100 150
BIAS 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
TRAINABLE PARAMETERS 76 76 76 76 76 76 101 101 101 101 101 101 101 101 101 151

enter image description here

Improve this answer
$\endgroup$
1
  • $\begingroup$ Please consider adding more text/explanation to your answer as pictures/links can sometimes become broken or unable to be viewed in the future. $\endgroup$
    – Ethan
    Oct 7, 2019 at 1:36
0
$\begingroup$

The filter $0$ of C3 is generated from the filter $0,1,2$ of S2. The kernel size is $5x5$. The number of parameter is $5x5x3=75 + 1(bias) = 76$. This applies to other filters as well. So overall, there are $76 + 76 +...+101+151 = 1516$ parameters.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.