In a dataset consisting of 1,000 samples, it has been shown that a 70-30 split (i.e. 70% of the samples used for training, 30% for validation) will provide a good estimation of the test accuracy of the trained models. If the dataset size increases to 10,000 samples, what split would you suggest?
$\begingroup$
$\endgroup$
-
1$\begingroup$ Welcome to DataScienceSE. I don't think the first statement is true for any dataset, it's probably a rule of thumb. Do you have a source for this statement? $\endgroup$ – Erwan Jan 9 at 22:07
-
$\begingroup$ For small datasets with only 1000 samples I'd rather suggest (nested) CV instead of hold-oud validation since results may largely dependent on the split point and you do not make good use of all your data $\endgroup$ – Sammy Jan 10 at 14:25
$\begingroup$
$\endgroup$
The current approach use 70/30 or 80/20, the most used is 80/20 (train/test). However there is other things you should check, for example if you data is balanced. If your data is not balanced you might want to use undersample or oversample.
