1
$\begingroup$

I have a data set which is already divided into 10 folds with each fold having training,validation and test sets. I'm not able to understand how to apply 10 fold cross validation on this data set.

In general, if we want to apply k-fold cross validation on a data set, the procedure is as follows

enter image description here

In my case, the data set is already divided into 10 folds and each fold contains validation and test sets in addition to training set. It would be helpful if someone can guide me, how to 10 fold cross validation for this kind of data set.

$\endgroup$
  • 2
    $\begingroup$ Welcome to this site! If you want to do K-fold CV on these K folds, ignore the inner training-validation-test separations, do the CV, then report the test score. Otherwise, why you are not allowed to ignore the inner separations and merge them? The answer to this question is key and depends on your specific case. $\endgroup$ – Esmailian Mar 27 '19 at 16:11
0
$\begingroup$

In 10 fold cross-validation, you split your dataset into 10 sections, 9 of them are for train and one for test set (there is no validation set), for example, if your dataset is 100 samples, inside a loop, in the first fold (first loop iter), the model train on 90 samples and the rest 10 are for testing the model, and loop is continued until all the dataset is used for training and testing.

for more, see here

and in python, you can implement 10 fold cross-validation using sklearn library here

Now, because your dataset is already split into 10 fold, you have two choices:

1- The easiest way is to combine your dataset into one set then using a specific library to do the 10 fold cross validation for you.

2- write code by yourself to loop over your 10 fold data, in the first iter use the first section for testing and the rest 9 for the training, in the second iter, use the second section for testing, and the first and other 8 sections for training, the loop should continue 10 times until all the data is used for training and testing.

this is the idea behind 10 fold cross validation if this not applicable for your dataset, I think 10 fold is not good in your case.

| improve this answer | |
$\endgroup$
  • $\begingroup$ The data set is already split to 10 folds with each fold internally split into train,test and validation sets. In this case, how to apply 10-fold cross validation?. @honar.cs $\endgroup$ – Mr. NLP Mar 27 '19 at 16:38
  • $\begingroup$ The answer you told is applicable in the case where each fold doesn't have internal split into train, test and validation sets. $\endgroup$ – Mr. NLP Mar 27 '19 at 16:40
  • $\begingroup$ I can ignore the internal split and apply cv. Here the question is, "Is there any other strategy to handle these kinds of datasets"? $\endgroup$ – Mr. NLP Mar 28 '19 at 2:11
  • $\begingroup$ the question is updated, see if it can help you. $\endgroup$ – Hunar Mar 28 '19 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.