I'm trying to follow a paper on deep NN feature visualization using beautiful examples from the GoogLeNet/Inception CNN. see: https://distill.pub/2017/feature-visualization/
The authors use backpropagation to optimize an input image to maximizes the activation of a particular (Inception) neuron/feature, or entire channel.
For example, Inception
Layer 4a, Unit 11 is feature 12 of 192 from the
1x1 convolution path of Inception Layer 4a before
filter concatenation (see: https://distill.pub/2017/feature-visualization/appendix/googlenet/4a.html#4a-11).
For Layer 4a
1x1 convolution the shapes are:
# Layer 4a input: [14,14,480] output: [14,14,512] # 1x1 convolution kernel: [1,1,480] # total 192 kernels output: [14,14,192] # channels [0..192] of Layer4a output Layer4a slice: tf.slice( layer4a_output, (0,0,0), (14,14,192) ) # Layer4a Unit 11 layer4a_unit_11 = tf.slice(layer4a_output, (11,0,0), (1,1,1)) # numpy [11,1,1]
In a related article, the authors state (see: https://distill.pub/2018/building-blocks/) ,
"We can think of each layer’s learned representation as a three-dimensional cube. Each cell in the cube is an activation, or the amount a neuron fires. The x- and y-axes correspond to positions in the image, and the z-axis is the channel (or detector) being run."
Furthermore, they offer a diagram which super-imposes the
cube of Layer4a over the input image with the
(x,y) axis overlaying the image itself.
I understand that the
Neuron Objective is the input image that produces the highest activation for
Layer 4a, Unit 11 which can be found at
index=[11,0,0] of Layer 4a
output=[14,14,512]. In this case,
[1,1,480] kernel generates a feature map of
shape=[14,14,1] with a total of 196 activations.
kernel => channel or feature map and
activation => neuron or feature.
But what is the intuitive concept of the
(Positive) Channel Objective? In this example, Unit 11 sits in the same channel as
14x14=196 other neurons, but the channel objectives for all these neurons are different. If the optimized image for the Channel Objective maximizes the sum of neuron activations for
channel 0, (e.g.
slice=[14,14,0] of 192 1x1 convolutions or 512 total layer 4a channels) wouldn't it be the same for all 192 neurons in the same channel? Obviously, by the examples we see this is not true.
How does the Channel Objective relate to the Neuron Objective for Unit 11?
I understand that the
Neuron Objectiveis the input image that produces the highest activation for
Layer 4a, Unit 11which can be found at
index=[11,0,0]of Layer 4a
This is where my understanding went off the rails.
Layer 4a Unit 11 is actually channel/feature 12 of 192 for the 1x1 convolution. It is NOT the 12 of 196 neuron of channel 1. My fault for confusing
192 channels with
Instead, as mentioned the in answer,
Unit 11 is a single neuron in channel 11, usually located near the center, e.g. Neuron Objective is
(x,y,z)=(7,7,11) and Channel Objective is