Help me solve this problem!

A training dataset has 10 classes and each class has 1000 samples. The samples in each class are redundant. If you apply full batch gradient descent, it takes 1000 updates to reach a local minima (say LM). You now apply standard mini-batch gradient descent (MBGD) with batch size 50 where the batch contains 5 samples from each class.

(a) How many epochs do you need to reach the LM in MBGD? Assume that in both cases initial parameter values and the learning rate are the same. (b) For the above data, what would be the optimal batch size for which mini-batch gradient descent will take minimum number of updates (iterations) to reach LM?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.