Let say if I have a large dataset of 300k instances with 200 features, I want to reduce its size. Can I apply K-Fold technique to the 200 features then the trimmed dataset are applied with RSS to trim the instances? Its like K-Fold to reduce Features then RSS to reduce instances then I get a small-scaled dataset with the less number of features and randomized smaller instances. Can this be done?

  • $\begingroup$ Hi, what do you mean by using k-fold and RSS to reduce features and instances? $\endgroup$
    – Kane Chua
    May 25, 2019 at 8:19
  • $\begingroup$ I want to reduce dataset size using both method $\endgroup$
    – Azlan Muhd
    May 25, 2019 at 8:56

1 Answer 1


K-fold cross-validation is a technique you use when you have too little training data. You use it to expand your training dataset, leaving only a tiny bit for testing (for example, 5%). Then you use k-fold cross-validation to select a different 5% for testing each time, and a different 95% of the full dataset for training each time. That way, you can get a good idea of the performance of your model, even if you only kept 5% of the data apart for training.

This is just an example, it could be that 5% is fine, or even too much, to hold out for testing. It depends mainly on the size of your dataset, and in case of classification, for example, it matter if one class has very few examples, so that perhaps in your 5% for testing there may not be a single example of that class.

What you're trying to do is called dimensionality reduction. The go-to technique for that is PCA. Make sure you normalize your features first (subtract the mean and divide by the variance), and then drop features until you still have, for example, 95% of the variance of your data explained. If in the end you find that you cannot get good performance from your model, it could be that you have lost too much information. In that case, you may have to increase this to 99%, adding features back in.

RSS is ok to reduce the number of training examples you have in your dataset, but it is only necessary if the order of examples is not random already. If it's random already, then you can just train on the first 10%, or whatever you need, and save yourself the trouble.

  • 1
    $\begingroup$ Great explanation, this is what I'm looking for! $\endgroup$
    – Azlan Muhd
    May 25, 2019 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.