``Hierarchial features extraction'' in Multilayer Perceptron models

Question

I am referring to plain neural networks, MLPs. I got to read the paper by Glorot and Bengio (2010), Understanding the difficulty of training deep feedforward neural networks.

Therein I read an interesting statement: ``Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features''.

What does that mean in the framework of MLPs? As far as I could understand, such a features extraction and abstraction process is made feasible by the use of convolutional layers, hence more advancer Deep Learning models.

Answer 1

In MLP, you can imagine it as similar to finding features in different spaces. Like for n features you come up with k feature by using PCA, similarly here MLP is converting the n features into k* features in different space.