In "Elements of statistical learning" page 395 the authors state that, relative to R(θ), the regression/classification error function in a neural network such as a multi layer perceptron:
Typically we don’t want the global minimizer of R(θ), as this is likely to be an overfit solution. Instead some regularization is needed: this is achieved directly through a penalty term, or indirectly by early stopping. Details are given in the next section.
However in Backpropagation, when momentum regularization is described the figure 9 shows how the back propagation process should steer the error function towards its global minimum, avoiding any local minimum found along the way.
How come these 2 reasonings are compatible? The only thing I can think of is that we want to get the global minimum for each coefficient error (as computed during the backprop process) while not adjusting for the "global" error function. Am I interpreting this correctly?