3
$\begingroup$

As far as I understood it, the pooling layer doesn't learn anything. It has several parameters, most important its pool_size and stride (see Lasagne documentation for more), but none of those is learned.

Is it possible to learn these two (or one of those) parameters? Are there papers about it?

(I would guess that it is not possible to add this to the objective function in a meaningful way... but I'd like to see if people thought about it.)

$\endgroup$
1
  • 2
    $\begingroup$ The fundamental issue with this is that changing the pool_size affects the dimensionality of the output from that layer. Therefore weights learned above that layer would have to be either re-initialized to be of the same dimension, or up/downsampled to match its size. $\endgroup$
    – jamesmf
    Nov 16 '15 at 14:14
4
$\begingroup$

No it is not possible to learn those meta parameters from a data set using a learning algorithm. There is no way to calculate a gradient of any objective function with respect to pool_size and stride params. Even if there were, they are typically discrete values and relatively small integers, so cannot be updated in the same way as e.g. the weights are.

What you can do is search through different values looking for optimal ones, using cross-validation to pick the best. The size and stride meta parameters can also have a large effect on memory use and speed of learning, so it may be the case you are looking to find a good compromise. With stride in lower layers for instance, it may be more effective (in terms of training time required for similar level of accuracy) to have a higher stride value but increase the overall training set size by adding random x/y translations to the training set images.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.