Timeline for What is the difference between Gradient Descent and Stochastic Gradient Descent?
Current License: CC BY-SA 4.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 28, 2019 at 2:20 | comment | added | Dan D. | tks, this is clear! | |
| Aug 7, 2018 at 19:34 | comment | added | n1k31t4 |
I'd say there is batch, where a batch is the entire training set (so basically one epoch), then there is mini-batch, where a subset is used (so any number less than the entire set $N$) - this subset is chosen at random, so it is stochastic. Using a single sample would be referred to as online learning, and is a subset of mini-batch... Or simply mini-batch with n=1.
|
|
| Aug 7, 2018 at 15:51 | comment | added | Developer | thanks, Briefly like this? There are three variants of the Gradient Descent: Batch, Stochastic and Minibatch: Batch updates the weights after all training samples have been evaluated. Stochastic, weights are updated after each training sample. The Minibatch combines the best of both worlds. We do not use the full data set, but we do not use the single data point. We use a randomly selected set of data from our data set. In this way, we reduce the calculation cost and achieve a lower variance than the stochastic version. | |
| Aug 4, 2018 at 9:32 | history | answered | n1k31t4 | CC BY-SA 4.0 |