0
$\begingroup$

Reading an article about 1x1 convolution, I found this:

It should be noted that a two step convolution operation can always be combined into one, but in this case [GoogLeNet] and in most other deep learning networks, convolutions are followed by non-linear activation and hence convolutions are no longer linear operators and cannot be combined.

What do they mean saying "two step convolution operation can always be combined into one"?

$\endgroup$

1 Answer 1

1
$\begingroup$

I think the comment is true for any kind of network where the neuron has a linear transformation function and there is no activation. Convolution is just a special case of linear transformation.

Basically, if your first layer outputs linear combinations of your features, and the second layer outputs linear combinations of the first layer outputs, then the second layers outputs is a linear combination of the initial features.

In the case of convolution, the interesting thing is that the convolution product is associative. So that if you apply two kernels consecutively $K_2 * (K_1*X) $ it is equivalent to $(K_2 * K_1)*X $ so that you can calculate the combined operation $ K_1 * K_2 $ as a unique kernel easily.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.