1
$\begingroup$

i understand mathematically that deep learning has more than one hidden layer, whereas regular machine learning hs just one. is that right? if so, why and how is it better to have more than one layer, that give deep learning the edge over machine learning?

$\endgroup$
  • $\begingroup$ I’m not sure if there’s a consensus on how many layers is “deep”. More layers gives the model more “capacity”, but then so does increasing the number of nodes per layer. Think about how a polynomial can fit more data than a line can. Of course, you have to be concerned about over fitting. As for why deeper works so well, I’m not sure if there’s a theoretical proof of why, but many people have used it to achieve great results. $\endgroup$ – Joe Jun 7 at 13:36
1
$\begingroup$

You can think of Neural Networks (however deep) as an approximation of an ideal function. The more layers/nodes are available, the more the Neural Network successfully approximate that ideal function. There is a theorem which states it.

Let's say you have to recognize human faces in pictures: there is at least one (ideal and purely hypothetical) mathematical function that is able to do the job. Than, a Neural Network can approximate that mathematical function in a more or less precise way.

Now: if you have the collection of every possible picture which can exist in the world since ever and forever (a so-called complete set), you could choose whatever number of layers and nodes, in whatever configuration, and you will never fall in an overfit situation. Instead, the more layers/nodes are available, the better your network will perform. But in this limited real world, you can get just a tiny subset of that main set.

So you have to choose the optimal number of layers and nodes, given a configuration, to avoid to fall in the situation of overfit. The more data samples you have, the more you can add up layers and nodes to the configuration, with the result of having better performances, i.e. a Neural Network which better approximate the (ideal and purely hypothetical) mathematical function introduced above.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.