Difference between validation and prediction

Question

As a follow-up to Validate via predict() or via fit()? I wonder about the difference between validation and prediction. To keep it simple, I will refer to train, val and test:

Training data: Train model, especially find hyperparameters through GridSearchCV or similar Validation data: Validate these hyperparameters on "new" data? Test data: Make prediction on unseen data

My status so far:

• Split data: 60 % Training - 20 % Validation - 20 % Test
• Find hyperparameters on training data
• Fit again with best parameters on training data by using .fit(X_train, y_train, validation_data(X_val, y_val)).
• Check model on unseen data through .predict() or .evaluate().

Is this correct? Though using GridSearchCV do I have to split train manually into training and validation?

Answer 1

In the GridsearchCV function, there is a parameter, cv that tells you how many times cross-validation is performed. If you're performing cross-validation on the train set, all you need is a train/test set since the hyperparameters are being tuned without seeing the test set.

If you want to use your own validation set, see the link below. The link below that also has a good description of how K-Fold CV works in GridSearchCV.

Finally, gridsearch optimization is pretty slow and there are more advanced methods for hyperparameter tuning now, in case performance is an issue.

See this previous answer:

Difference between test and validation set

Preselected Validation Set

Gridsearch CV and KFold CV