3
$\begingroup$

Suppose I'm training a linear regression model using k-fold cross-validation. I'm training K times each time with a different training and test data set. So each time I train, I get different parameters (feature coefficients in the linear regression case). So I will have K parameters at the end of cross-validation. How do I arrive at the final parameters for my model?

If I'm using it to tune hyperparameters as well, do I have to do another cross-validation after fixing the parameters of my model?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Related: datascience.stackexchange.com/questions/93733/…. $\endgroup$
    – Erwan
    May 19, 2022 at 14:24
  • $\begingroup$ The standard approach is this: 0) split training/test set 1) run CV with all the combinations of parameters on the training set 2) select the best combination of parameters from the avg performance across folds 3) re-train the model using only the selected parameters on full training set 4) evaluate the model on a fresh test set. Step 4 is important, you shouldn't evaluate on the same data used for CV. $\endgroup$
    – Erwan
    Jun 6, 2022 at 18:27
  • $\begingroup$ @Erwan, suppose I'm training a linear regression. How are my final feature coefficients selected? Depending on my cross-validation fold, coefficients change, right? $\endgroup$
    – NAS_2339
    Jun 7, 2022 at 7:58
  • $\begingroup$ The models trained during CV should not be reused, they are used only to estimate performance on the corresponding test set for this fold. Once CV is done, one has to retrain a single model using the full training data (step 3 in my comment above). $\endgroup$
    – Erwan
    Jun 7, 2022 at 9:21
  • 1
    $\begingroup$ Yes, exactly. . $\endgroup$
    – Erwan
    Jun 7, 2022 at 11:38

1 Answer 1

Reset to default
5
$\begingroup$

Usually, the aim of K-fold cross-validation is to check how a model performs (both on average and how much it varies across folds) given some hyper-parameters setting. We then pick the "best" set of hyper-parameters.

Afterwards, we fix the hyper-parameters and train the model with full dataset to squeeze all the juice.

In the case where there is no hyper-parameters to tune e.g. simple linear regression, cross-validation can give you an estimate of how your model will perform. You then train a final model with all data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.