Max-pooling vs. zero padding: Loosing spatial information

Question

When it comes to convolutional neural networks there are normally many papers recommending different strategies. I have heard people say that it is an absolute must to add padding to the images before a convolution, otherwise to much spatial information is lost. On the other hand they are happy to use pooling, normally max-pooling, to reduce the size of the images. I guess the thought here is that max pooling reduces the spatial information but also reduces the sensitivity to relative positions, so it is a trade-off?

I have heard other people saying that zero-padding does not keep more information, just more empty data. This is because by adding zeros you will not get a reaction from your kernel anyway when part of the information is missing.

I can imagine that zero-padding works if you have big kernels with "scrap values" in the edges and the source of activation centered in a smaller region of the kernel?

I would be happy to read some papers about the effect of down-sampling using pooling contra not using padding, but I can't find much about it. Any good recommendations or thoughts?

Figure: Spatial down-sampling using convolution contra pooling (Researchgate)

Answer 1

Pooling (max/mean/etc) has two primary benefits: it significantly reduces computational complexity (at the cost of potentially important data) and it helps the network achieve spatial invariance by making the spatial relativity between features less relevant.

However, the spatial invariance achieved through pooling is not always ideal. For instance, given a CNN trained to identify images of automobiles, it would probably classify an image containing a hood/trunk, tires, windows, car doors, etc. as an automobile, regardless of the components' relative positioning.

On the other hand, I do not know of significant costs to using padding. Zero padding is critical for deep networks; without it, our volumes would rapidly collapse through the layers. It therefore helps to maintain desirable volume sizes and to preserve the border data. It also helps the geometry of CNNs to work out smoothly by preventing mismatched dimensions between layers.

You might find this paper interesting. Hinton and his team propose a new network called Capsule Network (CapsNet) that addresses the problems with pooling and provides an alternative solution for acquiring spatial invariance. One of the many exciting parts of their algorithm is that the location of features relative to one another is a primary asset in the classification process, bringing computers closer to a "true understanding" of objectness.