Suppose that at a layer $N$ within a CNN, my "image" is a 200$\times$200$\times$10 array. Thus, if I convolve such an array with, for example, 15 filters of size 3$\times$3$\times$10, I will end up with a new "image" whose shape is $A\times{B}\times{15}$ where $A$ and $B$ depend on the stride of the convolution.
I am thus wondering how such 3D (3$\times$3$\times$10 in my toy example) convolution filters are usually displayed as 2D images (i.e. sum or mean, ...)?
With CNNs, a common strategy is to visualize the weights. They are usually most interpretable on the first CONV layer which is looking directly at the raw pixel data, but it is possible to also show the filter weights deeper in the network.
If you want to have a look at the activation maps, you will have to display them 1 by 1 and they will be grayscale (e.g. you would display 15 AxB grayscale filters for your layer N). Check this site for more ideas: http://cs231n.github.io/understanding-cnn/
Having said that, it is unclear what you can expect to "see" on layers that are looking at a 200*200*10 cube and what action you can take afterwards.
EDIT - Additional link This article describe a new technique that tries to show the "meaning" included in the neural network layers: https://distill.pub/2018/building-blocks/ "For instance, by combining feature visualization (what is a neuron looking for?) with attribution (how does it affect the output?), we can explore how the network decides between labels like Labrador retriever and tiger cat."
I am thus wondering how such 3D (3×3×10 in my toy example) convolution filters are usually displayed as 2D images (i.e. sum or mean, ...)?
TL;DR The 3x3x10 image is displayed as 10 3x3 images
Longer Version
Say you start with a 32x32x3 image
You decide to use a filter of shape 5x5x3 with an output depth of 15 (with no padding and stride 1). i.e., you will convert the 32x32x3 image to a 28x28x15 image. What does this depth of 15 really mean? Each of the 15 dimensions represents one filters output.
i.e., when we apply an output depth of 15 for a 5x5x3 filter, we take 15 5x5x3 filters and apply them to the image. From them, we get 15 28x28x1 images.
Each of these 15 $28$$\times$$28$$\times$$1$ images are stacked on top of each other to make a 28x28x15 image.
So, to answer your question : the $N$$\times$$N$$\times$$15$ images are split into 15 NxNX1 images and displayed.
Note: you'll notice in many CNN visualisations as the depth of the convolutional pyranmid increases, the number of small images increases.
