When would a neural network be defined as a Deep Neural Network (DNN) and not a NN?

A DNN as I understand them are neural networks with many layers, and simple neural networks usually have fewer layer... but what a many and a few in numbers? or is there some other definition?

What are networks trained used Tensorflow, Caffee as such? I haven't (as far I know) seen anybody manually design a network with many many layers. They seem to promote their tools for creating DNN, but is it actually DNN if you only make a network with two layers?

up vote 8 down vote accepted

You are right. Mainly any network with more than two layers between the input and output is considered a deep neural network. Libraries like tensorflow provide efficient architecture for deep learning applications such as image recognition, or language modelling using Convolutional neural networks and Recurrent neural networks. Another thing to keep in mind, is the depth of the network also has to do with the number of units being used in the layer. Mainly, as your non-linear hypotheses get complex you will need deep neural networks.

  • The distinction in the vocabulary, in my understanding, comes from the fact that learning neural network with more than a few layers is computationally intractable with the methods used before DNN actually exist. DNN represented a new set of learning methods that allowed to bring NN to another level of complexity, with numerous layers. – Eskapp Dec 29 '16 at 21:34
  • 1
    The depth and width of network are independent of each other. Depth offers generalization while width offers memorization. Also, one crucial thing is that just increasing no. of layers doesn't make it "deep learning" ! If you are efficiently learning feature representations (like in autoencoders) and not manually handcrafting features (like in old POS tagging techniques), you are still performing deep learning. – Rajarshee Mitra Feb 25 '17 at 19:29

The depth and width of network are independent of each other. Depth offers generalization while width offers memorization. Also, one crucial thing is that just increasing no. of layers doesn't make it "deep learning" ! If you are efficiently learning feature representations (like in autoencoders) and not manually handcrafting features (like in old POS tagging techniques), you are still performing deep learning.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.