1
$\begingroup$

I would like to ask you what a good size of dataset would be for building a classifier. I know that there are datasets of 1000 obs and datasets of 1m obs. But I also read papers where classifiers were built on datasets of 300 obs. I think the size may affect the accuracy/precision of a classifier, but I am not sure 100% of that.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

It's impossible to answer this question in general, because the answer strongly depends on the content of the data. More precisely it depends if the relations between the features and the target class in the training data are sufficient for the parameters of the model to be estimated as accurately as possible. In the most simple cases a few instances might be enough, for example a linear regression model with one single feature would need only two "perfect" instances. Practically the following factors have a strong impact on the number of instances needed:

  • the number of parameters to estimate and the complexity of the model: a more fine-grained model needs more detailed information, i.e. more instances
  • the number of classes, because there are more possible combinations for the model to learn and because it usually implies a higher number of parameters as well (more complex model)
  • the number of features, for the same reason
  • the amount of noise in the data, because finding the general patterns is more difficult if there are lots of inconsistencies/errors in the data, so statistically more instances are needed to distinguish the effect of chance from the real patterns.

So the only way to check how much data is needed for a particular task and dataset is to do an ablation study, i.e. a series of experiments in which a model is trained every time with a different number of instances. For example if one has 1000 instances, they could try to train a model with 100, 200, 300,...,1000 instances. Then by plotting the performance of every model one can observe whether the curve becomes stable at some point: if yes, this point shows how many instances are needed. If not (i.e. the performance keeps increasing significantly), then more than 1000 instances are needed.

I think the size may affect the accuracy/precision of a classifier, but I am not sure 100% of that.

Definitely.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for your answer @Erwan. $\endgroup$
    – V_sqrt
    Oct 25, 2020 at 1:34
0
$\begingroup$

The amount of data you have will only limit the types of classifiers you can try out on the set. If you have 100 samples you might still be able to perform a Logistic Regression - but you can forget about a Neural Net (this would require 100,000+ samples).

Take a look at this for more information on how much data is generally needed for ML.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I think it's a bit misleading to suggest that neural networks require 100,000+ samples as some sort of definitive answer. Also, I think the suggestion towards logistic regression needs to be more specific. Logistic regression with 100 features and no regularization might be an issue for example. $\endgroup$
    – aranglol
    Oct 24, 2020 at 2:07
  • $\begingroup$ @aranglol you're right. The point I am trying to make is some estimators require more data than others. There is no definite answer, as you say. As for the logistic regression example - I'm simply saying it can work (not that it will). $\endgroup$
    – Oliver Foster
    Oct 24, 2020 at 2:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.