Timeline for Why should the data be shuffled for machine learning tasks
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 22, 2022 at 7:23 | comment | added | Wojciech Jakubas | Should we shuffle between every epoch the whole data set, so we will end up with different training set every time, or should we shuffle just training set so we will end up with the same set but only different order? | |
| Sep 25, 2021 at 16:21 | comment | added | paperskilltrees | How come shuffling decreases the estimator's variance? | |
| Nov 9, 2017 at 19:13 | history | edited | Valentin Calomme | CC BY-SA 3.0 |
added 21 characters in body
|
| Nov 9, 2017 at 19:11 | comment | added | Josh | By shuffling, we are less likely to converge to a solution lying in the global minimum for the whole training set (higher bias), but more likely to find a solution that generalizes better (lower variance). | |
| Nov 9, 2017 at 19:10 | comment | added | Josh | I think shuffling decreases variance and is likely to increase bias (i.e., it reduces the tendency to overfit the data). Imagine we were doing full-batch gradient descent, such that epochs and iterations are the same thing. Then there exists a global minimum (not that we can necessarily find it) which our solver is trying to locate. If we are using MSE loss, then we will minimize bias if we could reach this solution every time. But since this global minimum is likely to be found in a different place for different training sets, this solution will tend to have high variance. | |
| Nov 9, 2017 at 13:19 | comment | added | Valentin Calomme | As I explained, you shuffle your data to make sure that your training/test sets will be representative. In regression, you use shuffling because you want to make sure that you're not training only on the small values for instance. Shuffling is mostly a safeguard, worst case, it's not useful, but you don't lose anything by doing it. For the stochastic gradient descent part, you again want to make sure that the model is not the way it is because of the order in which you fed it the data, so to make sure to avoid that, you shuffle | |
| Nov 9, 2017 at 12:38 | history | answered | Valentin Calomme | CC BY-SA 3.0 |