According to me:
Mini Batch Gradient Descent :
1.It takes a specified batch number say 32.
2.Evaluate loss on 32 examples.
4.Repeat until every example is complete.
5.Repeat till a specified epoch.
Gradient Descent :
1.Evaluate loss for every example.
2.Update loss accordingly.
3.Repeat till a specified epoch.
My questions are:
1.As Mini batch GD is updating weights more frequently shouldn't it be slower than normal GD.
2.Also I have read somewhere that we estimate loss in SGD (ie. we sacrifice some accuracy in loss calculation for speed). What does it means and does it helps in increasing speed.