After having read some theory I am getting a bit confused about the following terms:
- Deep Learning
- Deep Neural Network
- Artificial Neural Network
- Feedforward Neural Network
So, what seems clear to me is that Deep Neural Networks are Artificial Neural Networks with multiple layers (usually more than 1 hidden layer). However I have read several times that:
"Deep Neural Networks are feedforward Neural Networks with many layers."
I know what a feedforward Neural Network is, but to my understanding Deep Neural Networks is a term for ALL Artificial Neural Networks with multiple layers between input and output layer? Shouldn't there also be for instance Deep Recurrent Neural Networks? Is it correct that Deep Neural Networks must feedforward Neural Networks? This would in turn mean that Deep Recurrent Neural Networks cannot be referred to as Deep Neural Networks.
Moreover, I see there is a large variety of deep learning architectures, such as:
Convolutional Neural Networks
Residual Neural Networks
Deep Belief Networks
Deep Boltzmann machines
However, now also Wikipedia gives me a hard time to distinguish all the terms, saying: "Deep learning architectures such as deep neural networks, deep belief networks, recurrent neural networks and convolutional neural networks have been applied to fields including..."
So, my concrete questions rising from all the thoughts above are:
Does the term Deep Neural Networks ONLY belong to feedforward Neural Networks?
If the answer to 1) is yes: is the Wikipedia definition correct as you can read it up there? This would mean that i.e. a Convolutional Neural Network with multiple layers must be referred to as a Deep Convolutional Neural Network, which is not a subclass of Deep Neural Networks?
Is the term Deep Neural Networks a collective term for ALL Artificial Neural Networks with multiple layers, or just for all feedforward Neural Networks with multiple layers?
Would it be more accurate to use the collective term "Deep Learning Architectures" with strictly separated subclasses as Wikipedia suggests?