1
$\begingroup$

I want to build a classifier for my problem statement and for that I don't have data. So while doing data acquisition, what should be the minimum sample size? And would it be a good practice if I label each observation myself to build a valid data set? (I cannot automate the process of labeling observation to each class while doing data acquisition and manually doing that takes up lot of time)

$\endgroup$
2
  • $\begingroup$ It is difficult to answer without knowing anything about your problem, your objectives. So as for labelling, if it is about human labeling, you may be interested in Amazon Mechanical Turk --mturk.com/mturk/welcome $\endgroup$ – Eric Lecoutre Jun 22 '16 at 11:45
  • $\begingroup$ I want to classify my search results obtained in a search engine as relevant or not(1/0) based on the query entered. So, to build my classifier what should be the minimum size of my data set to get good results. $\endgroup$ – cgkentrus Jun 22 '16 at 11:51
2
$\begingroup$

Unfortunately you're not going to be able to do much without at least 200-300 records. You're going to be limited to simple (i.e. mostly linear) models until your dataset expands to at least 1,000. Anything less than 1,000 will require very thorough cross validation, and if you're not careful you'll be at risk of building a model that easily overfits.

@EricLecoutre makes a great point that you should use Amazon's Mechanical Turk. It usually costs just a penny or two per record and could save you a lot of time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.