I just studied K-Fold cross validation technique for finding model parameters and something seemed to be very confusing. Every tutorial I follow says that for K-Fold validation, the whole dataset will be split into K portions and K models will be fit with one portion as validation dataset each time. So, my doubt is that this method will generate K models. Which model are we supposed to use during actual inference? Or is it that same model will be trained K times with different portion of data as hold-out set?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
What you want to achieve with this validation strategy is a robust estimate of what combination of hyperparameters is good enough for your final model, so:
for each combination of hyperparameters, you carry out k trainings (as you ask with your last question), following this schema:
source of infoonce you have this k-trained models (i.e. for one hyperparams combination) you find the mean and standard deviation of the k sub models
you repeat this process for all the hyperparameters combinations you want to try out
Once you select the model hyperparams which provided the best mean value (and std) of the desired evaluation metric, you retrain with all the training dataset (the green section in the image), and evaluate it with the never-seen-before blue test data.
Luckily for you, this whole process is automated with helpers like this one from scikit-learn, which already retrains the final model for you, and is accessible via the best_estimator_ attribute.
If you want a bit more detailed documentation about this cross-validation, you can have a look at this link.
