4
$\begingroup$

Cross validation Vs. Train Validate Test

I have one further question relating to @louic answer in the post above:

"Training happens k times, each time leaving out a different part of the training set. Typically, the error of these k-models is averaged. This is done for each of the model parameters to be tested, and the model with the lowest error is chosen."

When we say "the model with the lowest error is chosen" which of the following does this mean:

  1. we choose the parameters which has the lowest averaged error and then find the the model with the absolute lowest error from within the K models
  2. we choose the parameters which has the lowest averaged error and then train a new model - on the entire training set, using these parameters?
  3. we choose the parameters which has the lowest averaged error and then take the average weights of the K-models trained Thanks

Iain

$\endgroup$

1 Answer 1

2
$\begingroup$

The answer in the linked question says to use approach 2. Specifically:

  1. Run cross-validation on the train set separately for each choice of hyper-parameter values.
  2. On each run, average the score across the k folds to produce a final cross-validation score.
  3. Choose the hyper-parameters that give us the best cross-validation score.
  4. Retrain a model with those hyper-parameters on the entire training set.
  5. Test this model on the test set, which has not been used at all until now.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.