Question about “1x3 and 3x1 conv is equivalent to 3x3 conv”

I see a lot of sites talk that we can substitute 1x3 conv + 3x1 conv for 3x3 conv.

In order to demonstrate easily, we use a 3x3 image as an example.

From the point of view of parameters, I know that we will get fewer parameters and end up with the same result.
If the image(3x3) perform a 3x3 conv, we will get a 1x1 scalar, and this kernel size is 3x3, means we have 9 parameters.
On the other hand, if the image first performs a 1x3 convolution, and then performs a 3x1 convolution on its output. It will intuitively get the same result, a 1x1 scalar, and this kernel size is 2x3x1 = 6, which means we have 6 parameters.

However, from the point of view of calculation time, I don't think 3x3 should be substituted by 1x3 & 3x1.

For example, if we use the image to perform a 3x3 kernel, we should do 9 times of pixel-wise multiply operations, and then one time of addition. Totally 10 operations to get the result.

But, if we use the image(3x3) to perform a 3x1 kernel, we have to implement [3(multiply) + 1(add)] * 3 times to get a 1x3 feature map, then we still need to perform a 1x3 kernel. It will then takes 3(multiply) + 1(add) operations. Totally 16 operations throughout the whole process.

What I want to show is that although we reduce the parameters from replacing 3x3 conv to 3x1 & 1x3 conv, we still need more time to calculate during the training process.

Training spends lots of time and implements 3x1 & 1x3 doesn't save our time. So, should we substitute 1x3 conv + 3x1 conv for 3x3 conv?

• @MatthieuBrucher Could you elaborate more on what diagonal filtering means? You want to say that if we use 3x1 followed by 1x3 filter (instead of 3x3) we are unable to extract diagonal features? please share some links and resources if possible – bit_scientist Sep 23 '20 at 5:08