Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I found this very informative figure, on how to split the dataset depending on how much data (or how many observations to be more precise) you have.
My question is, since "less data" is very subjective, is there a statistical test you can perform or even a rule of thumb on which split to follow?
My current problem is a classification problem with 145 observations, 22 features, 2 labels (18-True, 127-False), but I'm interested on the general approach.
Thank you.
As usual in these cases, there is no magic wand to determine which splitting method to use. It all depends on your specific data. Is your collected data redundant enough so that k-fold cross validation is not necessary? Does your data capture most of the input space?
Now, taking a look at your numbers, I'd say that the number of observations you have (145) is not likely to be large enough to capture all the potential variability in the input space, giving the fact that you have a high number of features (22). This conclusion, of course, depends on the type of the features (are they binary? categorical? numerical?), and whether all these features are actually necessary to make a prediction (are there redundant/correlated features? features that do not give any information about the output variables?).
In your case, and not knowing more about the data, I'd go for a cross-validation splitting scheme.
