I am currently studying the backpropagation process and gradient decent algorithm form the book Neural Networks and Deep Learning written by Michael Nielsen and 3Blue1Brown channel in YouTube.
My question is about calculating the gradient in gradient decent algorithm(the whole dataset as input).
For example we have 1 million hand written digit images and through the first iteration we feed the network with this 1 million images. Then the gradient is calculated for each images, summed together and averaged before updating the weights.
If my understanding is not wrong, this is the same thing that I saw in 3Blue1Brown channel. In this process the average of gradient is calculated with respect to the cost of each image and not the average cost of the whole data set(1 million) in one iteration, so the formula for calculating the cost has no effect here, rather its derivative is used for calculating the gradients for each image and we do not take the average of costs here.
First I want to know if this is a correct picture about how one iteration of calculating gradient decent works, second, why don't we take the derivatives of average costs with respect to the weights and biases and then take the average sum of all gradients?
And the last question, how the number of iterations and epochs can be assigned here? can we say number of epochs is always equal to the number of iteration since the whole data is used for each iteration?