0
$\begingroup$

What does one mean when they are talking about the simplicity of the networks? Does it mean that the shallower the networks the simpler they are, or does it mean that lesser the number of trainable parameters, simpler the models?

$\endgroup$
  • 1
    $\begingroup$ Could you clarify the link between your title and your question? $\endgroup$ – Leevo Dec 1 '19 at 9:27
  • $\begingroup$ I want to understand whether the depth of pre-trained neural networks makes them complex or the number of parameters. Sorry for the misleading title. I will change it. $\endgroup$ – Piyush Dongre Dec 1 '19 at 23:00
0
$\begingroup$

It can mean both, depending on the context. In order to improve prediction, a Network might need either more parameters or more depth, but these features improve models in different ways. For example, it's true that a Network with too few parameters won't be able to learn much; there is a minimal amount of parameters required simply to transform and represent signals in a sufficiently sofisticated way. On the other side, depth is fundamental in order to make your model generate more complex abstractions that will make it superior to other ML algorithms.

In other words: the number of parameters overall tells you how many things your network can learn. Its depth tells you how complex and sofisticated these things can be.

Deep Neural Network complexity is a multifaceted thing, it can't be represented on a single dimension.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.