# Cross validation techniques

What are the advantages vs disadvantages of cross validation types? Like k-fold, leave one out, etc.

• LOOCV may not work well for a data set with many samples. In the case, you would need to increase the amount you leave out – GK89 Sep 17 '16 at 1:04
• @GK89 why wont it work well? – Armon Safai Sep 17 '16 at 1:05

As you probably know by now, cross-validation is a method of:

1. partition dataset into 'train set' and 'test set'
2. fit model to train set, get the prediction error on test
3. repeat (1-2) in a bunch of different, 'pre-defined' ways, storing the prediction error each time.
4. average all the prediction errors to understand how your model will behave in the wild!

All the 'magic' of different cross-validation (CV) types happens at (3), basically. Say your dataset has $n$ observations.

• leave $p$ out CV separates your dataset into a train set of size $n-p$ and a test of $p$, gets the prediction error, and repeats this process for every single possible $p$-sized subset of the dataset.
• leave $1$ out CV is a special case of leave $p$ out where $p=1$
• $k$-fold cross CV partitions your dataset into $k$ equal sized partitions, trains on the first $k-1$ partitions and tests on the last partition, and gets the prediction error. The result at the end is ten $k$ prediction errors to average over. (Note that $k=n$ is the same as leave $1$ out CV.)

As for disadvantages and advantages, it's basically just a trade-off between exhaustiveness and speed. And I guess it depends on your dataset. The upper limit of exhaustiveness is basically doing leave $p$ out CV for all $p=\{1,2,...,n\}$. Probably depends too on the linearity of your dataset. In my experience, I haven't had trouble using $k$-fold cross CV.

Reference