Timeline for What is the difference between Gradient Descent and Stochastic Gradient Descent?
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 13, 2021 at 12:08 | comment | added | pegah | @JoshuaOlson: Correct! | |
| Sep 11, 2021 at 20:45 | comment | added | Joshua Olson | So I'm sure I get this right. The stochastic part just refers to "randomly" selecting a value out of all values to be representative for the update? | |
| May 11, 2021 at 23:32 | comment | added | haneulkim | @Sociopath Great explanation! I've tried both GD and SGD and GD performs better however I am not really understanding the reason behind it. do you know why? and if yes, is moving from SGD -> GD a way to mitigate underfitting ? | |
| Jan 24, 2021 at 12:14 | history | edited | Sociopath | CC BY-SA 4.0 |
added 43 characters in body
|
| Jan 17, 2021 at 18:17 | comment | added | Eric Cousineau | Note that the above link to cs229-notes is down. However, Wayback Machine, aligned with date of post, delivers - yay! web.archive.org/web/20180618211933/http://cs229.stanford.edu/… | |
| Aug 7, 2018 at 15:50 | comment | added | Developer | thanks, Briefly like this? There are three variants of the Gradient Descent: Batch, Stochastic and Minibatch: Batch updates the weights after all training samples have been evaluated. Stochastic, weights are updated after each training sample. The Minibatch combines the best of both worlds. We do not use the full data set, but we do not use the single data point. We use a randomly selected set of data from our data set. In this way, we reduce the calculation cost and achieve a lower variance than the stochastic version. | |
| Aug 4, 2018 at 6:39 | history | answered | Sociopath | CC BY-SA 4.0 |