When evaluating results using cross-validation, several strategies can be adopted, as using 5 or 10 folds, or doing leave one out cross-validation, as well as doing a 80/20 split.
Under which are general conditions should I try one or another?
$\begingroup$
$\endgroup$
asked Apr 23, 2018 at 20:20
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I generally advocate for cross validation in addition to a hold-out sample. As for the number of folds, that depends heavily on your data. Generally you start to approach diminishing returns after some point, but you should try, and evaluate, several regimes. This is very much an empirical question with no hard and fast best answer.
$\begingroup$
$\endgroup$
I think cross-validation is almost always superior to a simple train-test split. The only problem is computation time (for a 5-fold cross validation you have to train your model 5 times).
If your problem allows it, you should always use cross-validation.
The only reason you shouldn't use cross-validation is if your model takes too long to train (e.g. state-of-the-art image recognition networks require weeks of training on GPU clusters).