Each dense layer can approximate a function based on the neurons of the layer. The more neurons, the more accurate the function can be approximated (let's ignore overfitting). If so, what's the benefit to use multiple layers? The universal approximation theroem proposes, that each continues function can be approximated with a neural network with one hidden layer. Does every layer has a own approximation of the function? So, can one learn a discontinues function with adding layers to compensate for e.g. steep gaps or jumps? What is the reason, adding layers to a neural network can increase performance? Can anyone explain it in an intuitive way?
I got my information from this site. It does a good job of explaining the approximation part but not how multiple layers influence the neural network.
