Pr. Geoffrey Hinton has pointed out that pooling-layers remove spatial feature information. But, essentially, does the process that last convolutional layer's features are flattened for FC layer makes the spatial information removed?

  • 3
    $\begingroup$ Your title asks whether pooling layers remove spatial information (i.e. to confirm the information from Hinton's lecture). Your question in the body asks whether the flattening for fully connected layers removes spatial information. Could you make it clearer which question you want answered? $\endgroup$ Jul 21 '18 at 17:13
  • $\begingroup$ I just wonder if input image's spatial information disappeared at Fully connected layer (specifically, the process; last convolution layer's feature maps -> flatten -> Fully connected layer) $\endgroup$ Jul 22 '18 at 3:11
  • 2
    $\begingroup$ OK, that makes sense as a question. But why are you also asking about pooling layers, which are different? $\endgroup$ Jul 22 '18 at 8:30
  • $\begingroup$ I also know the spatial features disappear at pooling layer. I just want to know the spatial features can disappear at FC layer (through the above procedure). $\endgroup$ Jul 22 '18 at 11:34
  • 2
    $\begingroup$ OK, cool. But that is not what your question title says. Could you maybe edit it? $\endgroup$ Jul 22 '18 at 11:36

Your question text is not very clear but I try to give you what you need. Max-pooling layers give the CNNs spatial invariance ability through the layers due to the fact that they check the existence of something or not. If you stack them through multiple layers, your network will have a rough ability to be spatial invariance but this is not that much. Moreover, professor Hinton has stated somewhere that using pooling layers is a mistake and it's a disaster that they work; I didn't quote the exact words. If you want a network to be spatial invariance you should use spatial transformers which are differentiable modules and can be used in CNNs without any supervision. Take a look at here.

Finally, to answer your question, pooling layers somehow remove spatial information that is why capsnets are introduced.

About the fully connected layers, There are two main points. First the inputs, second, the outputs. In convolutional layers, the inputs of each layer is limited to the region which its is employed and for that input there is a single output. Consequently for a specific region, there is a single output which is responsible for that point. In MLPs, all the inputs go to a single neuron and for all the inputs there is just a single value. I guess this is why they don't keep the spatial information and are just used for classification tasks. Actually, they just try to classify the extracted features by the CNNs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.