8
$\begingroup$

When would a neural network be defined as a Deep Neural Network (DNN) and not a NN?

A DNN as I understand them are neural networks with many layers, and simple neural networks usually have fewer layer... but what a many and a few in numbers? or is there some other definition?

What are networks trained used Tensorflow, Caffee as such? I haven't (as far I know) seen anybody manually design a network with many many layers. They seem to promote their tools for creating DNN, but is it actually DNN if you only make a network with two layers?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
10
$\begingroup$

You are right. Mainly any network with more than two layers between the input and output is considered a deep neural network. Libraries like tensorflow provide efficient architecture for deep learning applications such as image recognition, or language modelling using Convolutional neural networks and Recurrent neural networks. Another thing to keep in mind, is the depth of the network also has to do with the number of units being used in the layer. Mainly, as your non-linear hypotheses get complex you will need deep neural networks.

Improve this answer
$\endgroup$
2
  • $\begingroup$ The distinction in the vocabulary, in my understanding, comes from the fact that learning neural network with more than a few layers is computationally intractable with the methods used before DNN actually exist. DNN represented a new set of learning methods that allowed to bring NN to another level of complexity, with numerous layers. $\endgroup$
    – Eskapp
    Dec 29, 2016 at 21:34
  • 1
    $\begingroup$ The depth and width of network are independent of each other. Depth offers generalization while width offers memorization. Also, one crucial thing is that just increasing no. of layers doesn't make it "deep learning" ! If you are efficiently learning feature representations (like in autoencoders) and not manually handcrafting features (like in old POS tagging techniques), you are still performing deep learning. $\endgroup$
    – Rajarshee Mitra
    Feb 25, 2017 at 19:29
0
$\begingroup$

The depth and width of network are independent of each other. Depth offers generalization while width offers memorization. Also, one crucial thing is that just increasing no. of layers doesn't make it "deep learning" ! If you are efficiently learning feature representations (like in autoencoders) and not manually handcrafting features (like in old POS tagging techniques), you are still performing deep learning.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.