I'm trying to follow a paper on deep NN feature visualization using beautiful examples from the GoogLeNet/Inception CNN. see: https://distill.pub/2017/feature-visualization/
The authors use backpropagation to optimize an input image to maximizes the activation of a particular (Inception) neuron/feature, or entire channel.
For example, Inception Layer 4a, Unit 11
is feature 12 of 192 from the 1x1 convolution
path of Inception Layer 4a before filter concatenation
(see: https://distill.pub/2017/feature-visualization/appendix/googlenet/4a.html#4a-11).
For Layer 4a 1x1
convolution the shapes are:
# Layer 4a
input: [14,14,480]
output: [14,14,512]
# 1x1 convolution
kernel: [1,1,480] # total 192 kernels
output: [14,14,192] # channels [0..192] of Layer4a output
Layer4a slice: tf.slice( layer4a_output, (0,0,0), (14,14,192) )
# Layer4a Unit 11
layer4a_unit_11 = tf.slice(layer4a_output, (11,0,0), (1,1,1)) # numpy [11,1,1]
In a related article, the authors state (see: https://distill.pub/2018/building-blocks/) ,
"We can think of each layer’s learned representation as a three-dimensional cube. Each cell in the cube is an activation, or the amount a neuron fires. The x- and y-axes correspond to positions in the image, and the z-axis is the channel (or detector) being run."
Furthermore, they offer a diagram which super-imposes the cube
of Layer4a over the input image with the (x,y)
axis overlaying the image itself.
I understand that the Neuron Objective
is the input image that produces the highest activation for Layer 4a, Unit 11
which can be found at index=[11,0,0]
of Layer 4a output=[14,14,512]
. In this case, (x,y)=[11,0]
. Each [1,1,480]
kernel generates a feature map of shape=[14,14,1]
with a total of 196 activations.
kernel => channel or feature map
and activation => neuron or feature
.
Question
But what is the intuitive concept of the (Positive) Channel Objective
? In this example, Unit 11 sits in the same channel as 14x14=196
other neurons, but the channel objectives for all these neurons are different. If the optimized image for the Channel Objective maximizes the sum of neuron activations for channel 0
, (e.g. slice=[14,14,0]
of 192 1x1 convolutions or 512 total layer 4a channels) wouldn't it be the same for all 192 neurons in the same channel? Obviously, by the examples we see this is not true.
How does the Channel Objective relate to the Neuron Objective for Unit 11?
ANSWER
I understand that the
Neuron Objective
is the input image that produces the highest activation forLayer 4a, Unit 11
which can be found atindex=[11,0,0]
of Layer 4aoutput=[14,14,512]
.
This is where my understanding went off the rails. Layer 4a Unit 11
is actually channel/feature 12 of 192 for the 1x1 convolution. It is NOT the 12 of 196 neuron of channel 1. My fault for confusing 192 channels
with 196 neurons/channel
.
Instead, as mentioned the in answer, Unit 11
is a single neuron in channel 11, usually located near the center, e.g. Neuron Objective is (x,y,z)=(7,7,11)
and Channel Objective is (x,y,z)=(:,:,11)