# How do I know if my training data set is sufficient?

If the number of classes is $C = 5$, training set contains $N = 3000$ samples, and $D = 4$ dimensions.

Is my data set is sufficient?

We can not judge whether our training data set is sufficient or not directly from the shape description for your data.

At the most of the time, we sample some instances, hopefully they are distributed as real-world data, so that the final model constructed through these data can reveal the truth of the real-world. Theoretically, the size of training data is not the problem, but we shall collect training samples as many as possible to ensure our model will have good performance on predicted instances.

Anyway, we can get the answer by testing whether the distribution of our predicted value is consistent with that of truth.

It depends of the model your training.

The data needs to be big enough so that you can subsample (with replacement) the data without loss of F1 score. That is the theory.

For Neural network, the rule of thumb is P^2, P being the number of parameters. This is often hard to achieve.

In practice, if you have the opportunity to create more data, make some, check the improvement, and if there is, you need more data (for your model, that is)

• That rule of thumb is intriguing, is there any intuition/heuristic for why $P^2$ would be good?
– Hugh
Nov 25, 2016 at 23:10