Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Reading an article about 1x1 convolution, I found this:
It should be noted that a two step convolution operation can always be combined into one, but in this case [GoogLeNet] and in most other deep learning networks, convolutions are followed by non-linear activation and hence convolutions are no longer linear operators and cannot be combined.
What do they mean saying "two step convolution operation can always be combined into one"?
I think the comment is true for any kind of network where the neuron has a linear transformation function and there is no activation. Convolution is just a special case of linear transformation.
Basically, if your first layer outputs linear combinations of your features, and the second layer outputs linear combinations of the first layer outputs, then the second layers outputs is a linear combination of the initial features.
In the case of convolution, the interesting thing is that the convolution product is associative. So that if you apply two kernels consecutively $K_2 * (K_1*X) $ it is equivalent to $(K_2 * K_1)*X $ so that you can calculate the combined operation $ K_1 * K_2 $ as a unique kernel easily.
