I have started reading Deep Learning Book, and I am having trouble understanding the advantages of RNN. This part of confuses me:
The unfodling process thus introduces two major advantages:
- Regardless of the sequence length, the learned model always has the same input size, because it is specified in terms of transition from one state to another state, rather then specified in terms of variable-length history of states.
- It is possible to use same transition function f with same parameters at every time step.
These two factors make it possible to learn a single model f that operates on all time steps and all sequence lengths, rather then needing to learn a separate model for all possible time steps. Learning a single shared model allows generalization to sequence lengths that did not appear in the training set, and enables the model to be estimated with far fewer training examples than would be required without parameter sharing.
I understand the 2nd advantage. Because the computations are recurrent, the input besides the current element in the sequence is the output of the previous hidden state which has the same structure as the current hidden state, thus the shared parameters.
But I do not understand the first advantage. I cannot visualize it or mathematically prove it, or at least I dont know how. Can anyone help me with this one? And if anyone has anything else to add on the difference ( advantages ) of RNN over ANN I would really appreciate it. Thanks in advance!