1
$\begingroup$

I have found the following definitions, but I don't really see the difference.

cross validation Method for testing classification and prediction models. The data are randomly split into N partitions (typically N=10) and then N times a model is created from N-1 partitions and tested on the "holdout" data.

Leave one out Every data point gets to be in a test set exactly once, and gets to be in a training set k-1 times.

$\endgroup$
1
$\begingroup$

Assuming your dataset includes $k$ samples:

In cross-validation, there are $N$ partitions, and the test split for each partition will have size $\frac{k}{N}$.

Leave-one-out validation is a special type of cross-validation where $N = k$. You can think of this as taking cross-validation to its extreme, where we set the number of partitions to its maximum possible value. In leave-one-out validation, the test split will have size $\frac{k}{k} = 1$

It's easy to visualize the difference. Here's two figures which contrast cross-validation and leave-one-out. In these figures, each sample in the dataset is represented by a colored circle. The training set is represented by the green circles, and the testing set is represented by the yellow circles.

5-fold cross validation:

5-fold cross-validation

Leave-one-out cross validation: leave-one-out cross validation

$\endgroup$
0
$\begingroup$

Consider a data set in which you have users and want to predict something about them based on some values. Each user might have many entries in the DS.

In cross validation, data is partitioned randomly so different users entries can fall into the same partition or some partitions might have no information about some user user, like below:

Dataset                  -> Partition

user1; somevalues        -> partition 1
user1; someOthervalues   -> partition 2
user2; somevalues        -> partition 2
user1; more_on_user1     -> partition 1
user2; someMoreValues    -> partition 2
user3; somevalues        -> partition 1 
etc...

partition 1 knows nothing about user2. Which might not be importan if you're assuming that behaviour is global an not particular to the user

Whereas leave-one-out will ensure that the test set has one example of each user. This is very common in recommendation systems in which for each user you leave one rating out, because a metric tested in all users is more reliable (in the context of rec.sys) than a metric that might skip many of them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.