I mean let's say I have a mini batch, I take an example from it and for it I do the following:
- I do forward propagation.
- Using the output after forward propogation - I calculate the gradients of the parameters.
Then I take the next example from the mini-batch and repeat steps 1 and 2 for it, and after I get the parameter gradients I sum them with the gradients I got earlier. After these actions are performed for all elements of the mini-batch, I use the resulting sum of gradients to find the average value of the gradients and use it to update the weights? Am I correct?