How can you decide the window size on a pooling layer?

Question

On the convolutional neural network, there used one or more pooling layers. As far as I know many tutorials instruct you to set it either 2 or 3 for the window size. For example, in this tutorial:

Pooling Layers

After some ReLU layers, programmers may choose to apply a pooling layer. It is also referred to as a downsampling layer. In this category, there are also several layer options, with maxpooling being the most popular. This basically takes a filter (normally of size 2x2) and a stride of the same length. It then applies it to the input volume and outputs the maximum number in every subregion that the filter convolves around.

I make the relevant part bold. But why is it usually set to 2 (or in some cases 3)? How can I know what size is appropriate? Should I just tweak the parameter via a trial and error like a brute-force way, or is there any tips on deciding the window size?

Answer 1

When you get a bit more insight into network topologies these hyperparameters will make more sense, but in general this is just like any other hyperparameter, you will have to test some settings and see what works better. In the case of pooling layers it is actually relatively interpretable. Why do we use pooling? To downsample our feature maps. This is done to decrease the resolution of our processed image but also to increase the receptive fields of our following layers. Because it is a form of downsampling we throw away all the information that was not the maximum in case of max-pooling, or we throw away information about the spread with mean-pooling. By increase our pooling filter size, we decrease our resolution even further and lose more information, but we will also need less parameters in our network if we have a fully connected layer at the end and our next convolutional layers will have a bigger receptive field, but with less attention to detail.