1
$\begingroup$

I have a dataset which is already divided into 10 folds with each fold having training, validation and test sets. I'm not able to understand how to apply 10-fold cross validation on this dataset.

In general, if we want to apply k-fold cross validation on a dataset, the procedure is as follows:

enter image description here

In my case, the dataset is already divided into 10 folds and each fold contains validation and test sets in addition to training set. It would be helpful if someone can guide me, how to do 10-fold cross validation for this kind of dataset.

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Welcome to this site! If you want to do K-fold CV on these K folds, ignore the inner training-validation-test separations, do the CV, then report the test score. Otherwise, why you are not allowed to ignore the inner separations and merge them? The answer to this question is key and depends on your specific case. $\endgroup$
    – Esmailian
    Mar 27, 2019 at 16:11

2 Answers 2

Reset to default
0
$\begingroup$

In 10 fold cross-validation, you split your dataset into 10 sections, 9 of them are for train and one for test set (there is no validation set), for example, if your dataset is 100 samples, inside a loop, in the first fold (first loop iter), the model train on 90 samples and the rest 10 are for testing the model, and loop is continued until all the dataset is used for training and testing.

for more, see here

and in python, you can implement 10 fold cross-validation using sklearn library here

Now, because your dataset is already split into 10 fold, you have two choices:

1- The easiest way is to combine your dataset into one set then using a specific library to do the 10 fold cross validation for you.

2- write code by yourself to loop over your 10 fold data, in the first iter use the first section for testing and the rest 9 for the training, in the second iter, use the second section for testing, and the first and other 8 sections for training, the loop should continue 10 times until all the data is used for training and testing.

this is the idea behind 10 fold cross validation if this not applicable for your dataset, I think 10 fold is not good in your case.

Improve this answer
$\endgroup$
4
  • $\begingroup$ The data set is already split to 10 folds with each fold internally split into train,test and validation sets. In this case, how to apply 10-fold cross validation?. @honar.cs $\endgroup$
    – Mr. NLP
    Mar 27, 2019 at 16:38
  • $\begingroup$ The answer you told is applicable in the case where each fold doesn't have internal split into train, test and validation sets. $\endgroup$
    – Mr. NLP
    Mar 27, 2019 at 16:40
  • $\begingroup$ I can ignore the internal split and apply cv. Here the question is, "Is there any other strategy to handle these kinds of datasets"? $\endgroup$
    – Mr. NLP
    Mar 28, 2019 at 2:11
  • $\begingroup$ the question is updated, see if it can help you. $\endgroup$
    – Hunar
    Mar 28, 2019 at 13:06
0
$\begingroup$

If your data is already divided into 10 fold you can try following :

  1. Take only training data from all the 10 splits

  2. Iterate i 10 times and each time take all the splits except the ith split as training and evaluate your model on the ith split

  3. Once done please take the average values of accuracy across all iteration to report cross validation scores

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.