SqueezeNet uses 1x1 convolutions. I try to understand it in this simple example: if the input is one MNIST digit, i.e. of shape 1x28x28x1
(I use Batch x Height x Width x Channel).
Then applying a Conv2D(16, kernel_size=(1,1))
produces an output of size 1x28x28x16
in which I think each channel 1x28x28xi
(i in 1..16) is just the multiplication of the input layer by a constant number. Is that right?
More specifically: Output[channel i][x,y] = InputLayer[x,y] * alpha_i
for x,y in 1..28, where alpha_i
is a constant for each channel.
Is this correct?
It's like going from 1 channel to 16 identical channels (except that they are multiplied by one global constant per channel).
What is its purpose?
Note: I have already read How are 1x1 convolutions the same as a fully connected layer? and 1x1 Convolution. How does the math work? but here it's slightly different.