When using K-Fold CV, is it still useful to have a Train/Validation/Test split?
Or simply just a Train/Test? I.e. split up data into k bins, and leave one out for testing, train on the rest, and take average of the scores.
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
It depends. If you're evaluating your model's performance without tuning hyperparameters, then a train/test split is sufficient.
If you're tuning hyperparameters, then you need a validation set. Within each fold, you'll train on the training set (of course), using the validation set to tune hyperparameters. Then you'll evaluate performance on the test set.