As I know, Gradient Descent has three variants which are:
1- Batch Gradient Descent: processes all the training examples for each iteration of gradient descent.
2- Stochastic Gradient Descent: processes one training example per iteration. Hence, the parameters are being updated even after one iteration in which only a single example has been processed.
3- Mini Batch gradient descent: which works faster than both batch gradient descent and stochastic gradient descent. Here, b examples where b < m are processed per iteration.
But in some cases they use Stochastic Gradient Descent and they define a batch size for training which is what I am confused about.
Also, what about Adam, AdaDelta & AdaGrad, are they all mini-batch gradient descent or not?