How much data should we use during training, and how much in testing? Can anyone explain why does it always seem to be 70:30 or 80:20 ratios?
Those rule of thumbs for ratio make no sense. The use case for a test set is to measure your performance in a real scenario on unseen data. How much you need for an accurate measure depends on the level of accuracy you need, the amount of variance you expect, not so much on the size of the original dataset. Something to be aware of is that you do decrease the size of your training set, which is a cost of testing. But when you have 10 million rows of data it makes no sense to validate on 3 million, when 10,000 could be more than enough. Outside of the cost of losing training data you should look at absolute numbers in your test set.
As far as reason behind train/test classification is concerned..you can have some help here
You and others are correct to question these "rules". Truth is, choosing a test/training split is a simple question with a complicated answer. See this paper which discusses different cross-validation methodologies, where
As a conclusion, guidelines are provided for choosing the best cross-validation procedure according to the particular features of the problem in hand.
In particular, they found the signal-to-noise ratio of the data had a lot to do with how the model should be trained and tested.
You can also find empirical papers (here's my dataset, here's my goal, here's what worked best) such as this one which found 10-fold cross-validation was best for model selection with that particular data set.
While I don't claim to have read everything, I haven't yet stumbled upon some who found a simple 80/20 or 70/30 split was the best way to do anything.
The proportion of your data split is arbitrary.
Things to remember however: the more training data you have, the better your model will be. The more testing data you have, the less variance you can expect in your results (ie. accuracy, false positive rate, etc.).
Ideally if you had infinite training data, you would be able to maximize the model's performance. The model would be the best it would ever be. And if you had infinite testing data you will be 100% confident in the resulting accuracy of your model.
Now, if you do not have sufficient testing data, for example you only have 1 example. Then it is possible that for the one example your algorithm correctly identifies its class and thus you assume that you have a 100% accuracy. But that would be wrong. You need multiple testing examples in order to correctly assess the performance of your model.
The 80:20 split is popular. As well as the 90:10. It's arbitrary. It all depends on how much data you have at hand. It also depends on how much data you expect to be sufficient to accurately train your model. If you only have 100 examples and you are training a data intensive model such as an NN then a 90:10 split is probably better. Although you will have high variance in your accuracy but your model will generalize better due to it having more data to train with.
I usually stick to 80:20 unless the dataset suggests otherwise. Also, it is good practice to use a validation set. If you have sufficient data, it is better to do 60:20:20. Train on 60% of the data, validate your model and tweek it on 20% of the data and when you are ready to submit your model test it on the final 20% of the data.
Cross-validation is also a good technique. You split your data into n bins. You then do leave-one-out training. You train on all the data bins except for 1, use the remaining bin to test. Repeat this procedure n times and take the average of your accuracies. Be wary however that you are spoiling some of your data by doing this since you are making use of all the data. It's always better to keep a chunk of the data for the final product, untouched until the last minute.