3
$\begingroup$

I feel this is a question that has a lot of variations already posted but it doesn't exactly answer my question. I understand the concept of max pooling and also the concept of backpropagation. What i cannot understand is that why is it said that max pooling doesn't affect the process of backpropagation? Is it because max-pooling "doesn't have any parameters"? Also is this statement - max pooling has no parameters- right? Why or why not?

$\endgroup$

2 Answers 2

2
$\begingroup$

Pooling layers does not have parameters which affect the back propagation. Backpropagation is an algorithm to efficiently implement gradient descent in a neural network by using the chain rule and propagating the error from the final layers to the early ones. The parameters that affect backprop are often called trainable variables (in Tensorflow), and they are the weights and biases of the convolutional filters. Adding pooling layers does not change the number of them.

Pooling layers are a way of performing downsampling, and they are used for the following main reasons:

  • To decrease the computational load of the network: smaller outputs means less FLOPS
  • To save memory during training: less memory allocation in the GPUs
  • To increase the receptive field of the neurons in successive layers. You can check this link about this topic: https://medium.com/@santi.pdp/receptive-fields-in-convolutional-neural-networks-6368a699d838
  • Adding transalation-invariance to the feature maps: when pooling, small translations of a feature will result in this feature located in the same position in the following pooled layer

What is true is that pooling layers may have a great impact in the learning capabilities of a network

$\endgroup$
2
  • 1
    $\begingroup$ This doesn't actually answer the question, you just gave an overview of a few relevant topics. It would be good to have some mathematical foundations in this answer $\endgroup$ Commented May 2, 2022 at 21:57
  • $\begingroup$ Fell free to add more content my friend. This is an open community, your contributions will be more than welcome ;) $\endgroup$
    – ignatius
    Commented May 3, 2022 at 22:09
0
$\begingroup$

This is a trick question. On the one hand yes, you don't have trainable parameters in max pooling - but this doesn't mean it has no affect on the backpropagation process. BUT - when thinking on the loss function - everything that influence the loss has an affect on the backpropagation. Specifically, max pooling modify the input and derivatives from trainable layers need to pass a changed-input-layer, the pooling layer - and take that into account. So, it does affect your derivatives calculation (aka backpropagation).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.