I was watching Andrew Ng's video on ResNets, and he mentioned that "And in theory, as you make a neural network deeper, it should only do better and better on the training set." Here is my understanding of neural networks, as the model progresses through the model, the parameters it will learn will become more and more sophisticated, correct? Intuitively, it should be able to recognize/discover more detailed pattern information about the training set. Is my understanding correct?
Then why, in practice, does adding excessive layers to Neural Networks actually harm the performance of the model? Thanks in advance.