5
$\begingroup$

Lets say you have a max pooling layer that gives 10 downsampled feature maps. Do you stack those feature maps, treat them as channels and convolve that 'single image' of depth 10 with a 3d kernel of depth 10? That is how I have generally thought about it. Is that correct?

This visualization confused me: http://scs.ryerson.ca/~aharley/vis/conv/flat.html

On the second convolution layer in the above visualization most of the feature maps only connect to 3 or 4 of the previous layers maps. Can anyone help me understand this better?

Related side question: If our input is a color image our first convolution kernel will be 3D. This means we learn different weights for each color channel (I assume we aren't learning a single 2D kernel that is duplicated on each channel, correct)?

$\endgroup$
  • 1
    $\begingroup$ It seems like this is some non-standard architecture; it is convolving several 3d kernels of different sizes with the first downsampled layer. Is there a description of this architecture somewhere? $\endgroup$ – eric.mitchell Nov 11 '17 at 1:50
  • $\begingroup$ Is what I describe the typical case then? Here is a link to the paper. Though I am mostly just wanting to understand what typically occurs. $\endgroup$ – Andrew King Nov 11 '17 at 1:55
  • $\begingroup$ That paper is kind of weird; it describes two different networks, neither of which seems to pertain to the network being displayed. $\endgroup$ – eric.mitchell Nov 11 '17 at 2:03
4
$\begingroup$

Lets say you have a max pooling layer that gives 10 downsampled feature maps. Do you stack those feature maps, treat them as channels and convolve that 'single image' of depth 10 with a 3d kernel of depth 10? That is how I have generally thought about it. Is that correct?

Yes. The usual convention in a CNN is that each kernel is always the same depth as the input, so you can also think of this as a "stack" of 2D kernels that are associated with the input channels and summed to make one output channel - because under the convention that $N_{in\_channels} = N_{kernel\_depth}$ this is mathematically the same. Expressing as a 3D convolution allows for simpler notation and code.

On the second convolution layer in the above visualization most of the feature maps only connect to 3 or 4 of the previous layers maps. Can anyone help me understand this better?

The diagram is non-standard in that respect, although it seems to show pooling and fully-connected layers as normal. It might be a mistake in the diagram, or something unconventional about that specific CNN.

If our input is a color image our first convolution kernel will be 3D. This means we learn different weights for each color channel (I assume we aren't learning a single 2D kernel that is duplicated on each channel, correct)?

Correct. You can see this in the visualised filters for AlexNet (do note that for computational reasons, AlexNet specialised one half of its filters to work in greyscale, and had other clever optimisations that we don't use nowadays because available GPU power is high enough to not need them). Most implementations will also treat a greyscale image as a 1-channel 3D shape for consistency.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.