In the context of Machine Learning, I encounter often the fact that a correction step does not occur after each training step, but only every n learning steps.
Citing from the Deep Learning with Python book:
This is the training loop, which, repeated a sufficient number of times (typically tens of iterations over thousands of examples), yields weight values that minimize the loss function
Why don't we correct at every step, but typically only once every 100 learning samples?
I assume, but I am not sure, that this might be because of efficiency, and also to smooth the correction "path" (e.g. integrating a correction step that is an average of the last 100 loss function values).
Thank you in advance!