4
$\begingroup$

From what I read online, nested CV works as follows:

  • I divide my whole data in k folds, where k-1 folds are the train set and one fold is the test set.
  • There is the inner CV loop, where we may conduct a grid search using the train set to find the best hyperparameters for a model.
  • There is the outer CV loop, where we measure the performance of the model that won in the inner fold, on the test set.
  • We repeat the above procedure for different test and train sets until, at some point, all folds got his place as a test set once.

what I cannot understand is that because we find the hyperparameters in each outer loop we run, we might have models with different hyperparameters being tested in the test set in each loop, so can we use this nested cross validation to find the best hyperparameters for a model? or the goal of this evaluation is to find the best algorithm, like SVM or Naive Bayes, for a data set? because, if we are getting different hyperparameters in each loop, we cannot say which one is the best one.

English is not my first language, if it my text is hard to understand, please tell me, so I can fix it.

$\endgroup$
1
$\begingroup$

Generally, you consider the outer CV as just estimating the performance of the method of fitting. You don't expect a final model out at this point. You can then apply the same method of fitting a model (what was happening in the inner CV) to the entire dataset to produce a model, and you say that you expect the performance estimate to apply to this new model.

https://stats.stackexchange.com/a/65158/232706
https://datascience.stackexchange.com/a/16856/55122
https://stats.stackexchange.com/q/11602/232706
https://stats.stackexchange.com/q/52274/232706

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.