I have a matrix of 358.367 data. Each row is a DNA sequence from the human genome. I want to build a classification model in R, using XGBoost algorithm and 83 features (dinucleotides, trinucleotides, etc.).

How should I split the data for the train and test set?

For example 70% for the train set and 30% for the test set? 30% for the train set and 70% for the test set?


1 Answer 1


There is no "golden rule" here. Your data set is very handy - neither too large nor too small. Sounds like a very exciting project!

Here is how I often proceed in comparable settings.

  1. Do all splits stratified by response or, if the rows are not independent but rather clustered by some grouping variable (e.g. the family etc), grouped sampling. Important rule: avoid any leakage across splits.

  2. Set aside 10%-15% of rows for testing. Don't touch them until the analysis is complete. Act as you would never utilize this test set.

  3. Select loss function and relevant performance measure.

  4. Fit a random forest without tuning and use its OOB error as benchmark.

  5. Choose parameters of XGB by 5-fold cross-validation iteratively by grid search, first starting with very wide parameter ranges and then making those ranges smaller and smaller. The number of boosting rounds are automatically optimized by early stopping.

  6. Choose model and present cross-validation performance.

  7. At the very last, reveal test performance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.