I'm confused by the concept of equating a 1x1 convolution with a fully connected layer. Take the following simple example of a 1x1 convolution of 2 input channels each of size 2x2, and a single output channel.
The only way I can relate this to fully connected layers is to say that there are 4 fully connected layers, one for each location in the input feature map (inputs and outputs colour coded).
From what I can understand my interpretation is consistent with the Network in Network paper[Lin et al. 2013] which describe the 1x1 as being equivalent as cross channel parametric pooling
The cross channel parametric pooling layer is also equivalent to a convolution layer with 1x1 con- volution kernel.
I have seen this one from Yann LeCunn equating 1x1 convolutions to a fully connected layer. And I have read this answer and I'm just not seeing the equivalence between a 1x1 convolution over an input volume and a single fully connected layer...
Any insight would be appreciated, if you can please relate back to the example above. Thanks!