A book I'm now reading, "Apache Mahout Cookbook" by Pierro Giacomelli, states that

To avoid [this], you need to divide the vector files into two sets called the 80-20 split <...> A good dividing percentage is shown to be 80% and 20%.

Is there a strict statistical proof of this being the best percentage, or is it a euristic result?

  • $\begingroup$ What do you do with that split? Can you give us more context on that? $\endgroup$ – rapaio Feb 11 '15 at 13:44

If this is about splitting your data into training and testing data, then 80/20 is a common rule of thumb. An "optimal" split (which would need to be operationalized) would likely depend on your sample size, distributions and relationships between your variables.

It is also common to split your data three ways (e.g., 60/20/20 - again rules of thumb), into a training set that you train your models on and a test set which you test your model on. You will iterate training and testing until you like the result. Then, and only then you apply the final model (trained on both the training and test set) on the third validation set. This avoids "overfitting on the test set".

However, cross-validation is much better than a simple data split. Your textbook should also cover cross-validation. If it doesn't, get a better textbook.

| improve this answer | |
  • $\begingroup$ So, rule of thumb it is. Thanks. As for cross-validation, I haven't read that far yet, so it might cover it. $\endgroup$ – Chiffa Feb 11 '15 at 14:15
  • $\begingroup$ As we know, 80/20 rule aka Pareto principle is based on on the Pareto distribution. Consequently, the 80/20 split is based on the specific value of the Pareto index ($\alpha \approx$ 1.161). What I'm curious about is whether the two-way and three-way data splits have an analytical solution, based on the parameters you've mentioned ("sample size, distributions and relationships between your variables") as well as Pareto distribution's parameters. $\endgroup$ – Aleksandr Blekh Feb 11 '15 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.