In K fold cross validation, we divide the dataset into k folds, where we train the model on k-1 folds and test the model on the remaining fold. We do so until all the folds were assigned as the test set. In every of these iterations, we will train a new model that is independent of the model created from the previous iteration ( every iteration uses a new instance of the model). So my question is, if I divide my dataset into train and test sets, then I have used only the training set for the k cross validation process, and since every iteration uses a new model, what is the output model from this k fold cross validation process that I should use to evaluate it ( calculates the ROC curve, F1-score, precision and so on) using the test set ?? (As I have different models for every iteration). One way to implement k fold cross validation is to use sklearn.model_selection.cross_val_score and this returns only an array of scores of the model for each run of the cross validation and this confirms my problem, where there is no model is returned to be further evaluated by the test set. What should I do in this case ?
1 Answer
$\begingroup$
$\endgroup$
One of the most common uses of k-fold cross-validation is for model selection (i.e. what type of model - such as linear regression, random forest, neural network) is best for the problem and/or what are the appropriate hyperparameter settings. We train models using K-fold cross-validation for each model type and/or set of hyperparameters we want to test, and select the best one based on the cross-validation results. Then we use all the training data to train another model, with the hyperparameters set to the values found by the cross-validation process. This last model is one evaluated using the test data.
