I would like to develop a machine learning algorithm using the knn model to perform a classification of my data records. My question is: is there a general method to follow to determine how large my training dataset should be?
is there a general method to follow to determine how large my training dataset should be?
No, there's no general method because it depends on the data: number of features, number of values for the features, diversity in the instances...
Instead there are indirect methods:
- The main problem when the training set is too small is the risk of overfitting. Thus if a model is overfit when trained with $N$ instances then it's an indication that $N$ is too low. Overfitting is usually detected when the performance on the training set is much higher than on the test set.
- More general: an ablation study consists in evaluating models trained with different number of instances, and observe their performance as a function of the size. If the performance becomes stable around size $N$ then this shows that $N$ instances is sufficient.