I have done a 10 fold Cross Validation on my data and have selected the best model from the results. With cross validation, I will have 10 models trained from different folds of the data. For the final model to use, should I take the average of the models or just fit a model for the entire training set?
$\begingroup$
$\endgroup$
Add a comment
|
$\begingroup$
$\endgroup$
Typically you would use the best model parameters and then re-run the model with the portion of the data set aside for training to come up with a new 'best' that you can run against your test set.
$\begingroup$
$\endgroup$
I will suggest you to read a post on K-Fold CV
Once we have mean score of each model, we generally select the best model out of it.
$\begingroup$
$\endgroup$
Cross validation is a procedure used to get a sense of model performance with the procedure (type of model, preprocessing, selected hyperparameters etc) you have selected. Let's suppose you have 2 models A and B and you want to know which to select. You would perform similar CV on both the models and avergae the score of both the CV to get an estimate of their performance.
Let's say A performed better. So you would select A and then train on your entire train set and then predict on test set. Same procedure goes for hyperparameter tuning.
You should take the average of the CV score you get. That score will be an "indicator" of your model performance and not the "final" model.
$\begingroup$
$\endgroup$
We usually take best model. But I have come across a notebook on Kaggle where the user has taken best two models from k-fold validation and then used both of them in ensemble learning. She has given 60% weight to the best model and 40% to the second best model. You can try that. Doesn't hurt, rt?
