5
$\begingroup$

Suppose that at a layer $N$ within a CNN, my "image" is a 200$\times$200$\times$10 array. Thus, if I convolve such an array with, for example, 15 filters of size 3$\times$3$\times$10, I will end up with a new "image" whose shape is $A\times{B}\times{15}$ where $A$ and $B$ depend on the stride of the convolution.

I am thus wondering how such 3D (3$\times$3$\times$10 in my toy example) convolution filters are usually displayed as 2D images (i.e. sum or mean, ...)?

$\endgroup$
  • $\begingroup$ How are you visualising your input data? $\endgroup$ – Neil Slater Feb 28 '17 at 14:13
  • $\begingroup$ If my input data are RGB images, using one channel per depth component. $\endgroup$ – floflo29 Feb 28 '17 at 14:35
  • $\begingroup$ I don't understand how you get 200 x 200 x 10 from that, nor how you visualise it. I am asking because if you have a way that you need to visualise the input data, it is likely to inform how you visualise the convolutional filters for that input data. Or, is the 200 x 200 x 10 picked from some layer N, and these are just normal images? CNNs are used for more than images, so perhaps make it clear in the question that your input data is 2D images. Use of 3D in the title implies it is something else . . . and if the x 10 part is channels, then you are doing 2D convolution and have 2D filters. $\endgroup$ – Neil Slater Feb 28 '17 at 14:44
  • $\begingroup$ The shape 200 x 200 x 10 is a random shape (just for example purpose) that could be the shape at a given layer within the convolutional neural net, not the very first input of the network. $\endgroup$ – floflo29 Feb 28 '17 at 14:48
  • 1
    $\begingroup$ Yes, as I said you have a 3D structure there, but it is a stack of 3 times 2D filters, not a 3D filter. It is a 2D convolution. $\endgroup$ – Neil Slater Feb 28 '17 at 15:03
1
$\begingroup$

With CNNs, a common strategy is to visualize the weights. They are usually most interpretable on the first CONV layer which is looking directly at the raw pixel data, but it is possible to also show the filter weights deeper in the network.

If you want to have a look at the activation maps, you will have to display them 1 by 1 and they will be grayscale (e.g. you would display 15 AxB grayscale filters for your layer N). Check this site for more ideas: http://cs231n.github.io/understanding-cnn/

Having said that, it is unclear what you can expect to "see" on layers that are looking at a 200*200*10 cube and what action you can take afterwards.

EDIT - Additional link This article describe a new technique that tries to show the "meaning" included in the neural network layers: https://distill.pub/2018/building-blocks/ "For instance, by combining feature visualization (what is a neuron looking for?) with attribution (how does it affect the output?), we can explore how the network decides between labels like Labrador retriever and tiger cat."

$\endgroup$
0
$\begingroup$

I am thus wondering how such 3D (3×3×10 in my toy example) convolution filters are usually displayed as 2D images (i.e. sum or mean, ...)?

TL;DR The 3x3x10 image is displayed as 10 3x3 images

Longer Version

Say you start with a 32x32x3 image

You decide to use a filter of shape 5x5x3 with an output depth of 15 (with no padding and stride 1). i.e., you will convert the 32x32x3 image to a 28x28x15 image. What does this depth of 15 really mean? Each of the 15 dimensions represents one filters output.

i.e., when we apply an output depth of 15 for a 5x5x3 filter, we take 15 5x5x3 filters and apply them to the image. From them, we get 15 28x28x1 images.

Each of these 15 $28$$\times$$28$$\times$$1$ images are stacked on top of each other to make a 28x28x15 image.

So, to answer your question : the $N$$\times$$N$$\times$$15$ images are split into 15 NxNX1 images and displayed.

Note: you'll notice in many CNN visualisations as the depth of the convolutional pyranmid increases, the number of small images increases.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.