# Can we use decreasing step size to replace mini-batch in SGD?

As far as I know, mini-batch can be used to reduce the variance of the gradient, but I am also considering if we can achieve the same result if we use the decreasing step size and only single sample in each iteration? Can we compare the convergence rate of them?

• As @mirror2image says - there's no free lunch for you. It depends highly on your data, domain, problem - run your experiments and check it. Generally speaking, rarely there's some significant a priori knowledge. Running many experiments will also aid your intuition for future research, as a space of hyper-parameters you can research is always limited. – Piotr Rarus Nov 25 '19 at 14:35