I'm not sure if I understand the purpose and generalization of 'nested cv' correctly.

I found information online that the purpose of nestd cv is to be able to correctly estimate generalization error. And we saw that generalization means that when external data that is different from the training data is input, similar results are obtained as if the model was trained using the training data.

I understood this meaning as follows. For example, as a result of nested 10 cv, the average AUC score in dataset A is train: 0.85, test: 0.78. If the average AUC score in data set B after removing feature 'a' from data set A is train2: 0.80, test2: 0.77

Even if the test score of data set B is lower than A, can it be said to be a good generalization because the difference between train and test scores is less than A?

Is it okay to use nested cv like this to evaluate how well a given model generalizes on a particular dataset?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.