1
$\begingroup$

If we consider Batch Gradient, Stochastic Gradient, Mini-batch Gradient, will they affect on the actual predictions? as we know they always attempt to reach a local minimum.

$\endgroup$
1
$\begingroup$

will they affect on the actual predictions?

Yes. The type of gradient descent that you use has an impact on how the model is trained. Therefore, it may lead to different predictions as the trained models may be different.

as we know they always attempt to reach a local minimum

Yes, each gradient descent technique has the same goal, which is to optimize a cost function by following its gradient. However, as you point out, gradient descent may end up in different local minima and some of them may be better than others. And each gradient descent technique is more or less prone to finding the global minimum, which may lead to a difference in predictions.


I would also recommend that you experiment with these different approaches and look at how the training progresses. This may help you gain intuition about the impact of each technique. Also, I find this blog to be quite useful to better understand the differences: https://towardsdatascience.com/gradient-descent-algorithm-and-its-variants-10f652806a3

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.