Cross validation Vs. Train Validate Test cont'd

Question

Cross validation Vs. Train Validate Test

I have one further question relating to @louic answer in the post above:

"Training happens k times, each time leaving out a different part of the training set. Typically, the error of these k-models is averaged. This is done for each of the model parameters to be tested, and the model with the lowest error is chosen."

When we say "the model with the lowest error is chosen" which of the following does this mean:

1. we choose the parameters which has the lowest averaged error and then find the the model with the absolute lowest error from within the K models
2. we choose the parameters which has the lowest averaged error and then train a new model - on the entire training set, using these parameters?
3. we choose the parameters which has the lowest averaged error and then take the average weights of the K-models trained Thanks

Iain

Answer 1

The answer in the linked question says to use approach 2. Specifically:

1. Run cross-validation on the train set separately for each choice of hyper-parameter values.
2. On each run, average the score across the k folds to produce a final cross-validation score.
3. Choose the hyper-parameters that give us the best cross-validation score.
4. Retrain a model with those hyper-parameters on the entire training set.
5. Test this model on the test set, which has not been used at all until now.