I am reading Francois Chollet's Deep learning with python, and I came across a section about max pooling that's really giving me trouble.

I simply don't understand what he means when he talks about "What's wrong with this setup?". How does removing the max pooling layers "reduce" the amount of the initial image that we're looking at?


2 Answers 2


Max pooling layers are useful for two things:

  • undersampling the dimensions of their input, to reduce model parameters and also reduce overfitting
  • increasing the robustness of the model to small changes in the input image (e.g. if I change the input image slightly the model should predict the same thing).

However their usefulness is limited and are starting to be replaced by convolution layers with large strides. You could read this article on the matter.


The max pooling, in effect, creates a short-hand to summarise the response map of each convolution. If we have a pooling window of 2 x 2 with stride 2, we are splitting the response map into 2x2 sections and asking the question, what is the best way to summarise what is going on in this small region? There are different options to do so, taking the maximum says that we look at the strongest response only. Taking an average takes all pixel responses into account. You can even take a median, though there are issues to do that computationally speaking. After pooling, lots of small summaries of the responses to all the various convolutions.

Every time you do pooling, you reduce the dimensionality of the problem. Remember that your end goal is to create a classifier on a vector that represents the features of the original input. If you have a very big vector you are dealing with a very high dimensional space which is computationally expensive to seperate out sets in.

Another advantage is that building higher level features requires finding relationships between pixels at more than a local level. The reason we use convolutions is because is makes sense to have relationships between pixels that are connected in some way and excluding distant relationships between pixels helps reduce the problem size. Repeated layers of response maps and summarising allow us to draw conclusions from a larger area as we start taking summaries of summaries. A 2 by 2 area after pooling really contains information from a 4 by 4 area of the original image and this multiplies exponentially with the number of layers. If we just did convolution without pooling, we cannot look beyond the initial area as there is no summarising going on.

Thirdly, we are trying to optimise the cost function over the space of weights - experiment with different levels of convolutions with and without pooling and fully connected layers and you will see how the number of weights has the potential to explode as we increase the depth of the network. Even using convolutions, we need pooling, but there is some debate about whether this is harmful to the end goal. Geoff Hinton says that CNNs lose too much spatial information - when we summarise, we don't care about where the pixel of greatest response comes from. Repeated layers make where the information comes from very jumbled. It can't tell the difference between a picture of a real person and a Picasso portrait of a person. He proposes Capsul e Networks, but as far as I know they have not achieved any notable success when compared with CNNs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.