# Which observation to use when doing k-fold validation or boostrap?

I have to perform predictive model over the dataset $D$ (with 1000 obs). From $D$, I extract 700 obs for training $(T)$ and 300 obs for validation $( V )$.

I need to perform bootstrap or 10-fold cross validation sampling.

The question is which of these sets should I use?

• Divide $D$ in 10 subsets and alternate training and validation between them ?

• Divide $T$ (the training subset) in 10 subsets and perform training/validation on those subsets? $V$ is used only for final validation.

I recommend using the second option you presented. I would use $T$ with 10-fold CV to select my modeling technique and optimal tuning parameters. Take a look at what performed the best ("best" being the model that gives us the best error, but also doesn't have the error fluctuate too much from fold to fold). After selecting a model, you can use the model on $V$ to get a realistic error rate.