3
$\begingroup$

I just studied K-Fold cross validation technique for finding model parameters and something seemed to be very confusing. Every tutorial I follow says that for K-Fold validation, the whole dataset will be split into K portions and K models will be fit with one portion as validation dataset each time. So, my doubt is that this method will generate K models. Which model are we supposed to use during actual inference? Or is it that same model will be trained K times with different portion of data as hold-out set?

$\endgroup$
4
$\begingroup$

What you want to achieve with this validation strategy is a robust estimate of what combination of hyperparameters is good enough for your final model, so:

  • for each combination of hyperparameters, you carry out k trainings (as you ask with your last question), following this schema: enter image description here source of info

  • once you have this k-trained models (i.e. for one hyperparams combination) you find the mean and standard deviation of the k sub models

  • you repeat this process for all the hyperparameters combinations you want to try out

Once you select the model hyperparams which provided the best mean value (and std) of the desired evaluation metric, you retrain with all the training dataset (the green section in the image), and evaluate it with the never-seen-before blue test data.

Luckily for you, this whole process is automated with helpers like this one from scikit-learn, which already retrains the final model for you, and is accessible via the best_estimator_ attribute.

If you want a bit more detailed documentation about this cross-validation, you can have a look at this link.

$\endgroup$
1
  • $\begingroup$ Precise answer. Thank you so much! $\endgroup$ – Ricky Apr 5 at 4:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.