why is more layers better in deep learning?

Question

i understand mathematically that deep learning has more than one hidden layer, whereas regular machine learning hs just one. is that right? if so, why and how is it better to have more than one layer, that give deep learning the edge over machine learning?

Answer 1

You can think of Neural Networks (however deep) as an approximation of an ideal function. The more layers/nodes are available, the more the Neural Network successfully approximate that ideal function. There is a theorem which states it.

Let's say you have to recognize human faces in pictures: there is at least one (ideal and purely hypothetical) mathematical function that is able to do the job. Than, a Neural Network can approximate that mathematical function in a more or less precise way.

Now: if you have the collection of every possible picture which can exist in the world since ever and forever (a so-called complete set), you could choose whatever number of layers and nodes, in whatever configuration, and you will never fall in an overfit situation. Instead, the more layers/nodes are available, the better your network will perform. But in this limited real world, you can get just a tiny subset of that main set.

So you have to choose the optimal number of layers and nodes, given a configuration, to avoid to fall in the situation of overfit. The more data samples you have, the more you can add up layers and nodes to the configuration, with the result of having better performances, i.e. a Neural Network which better approximate the (ideal and purely hypothetical) mathematical function introduced above.