To train a nice supervised algorithm (for instance, a dependency parser, a parts-of-speech tagger or NER) data is essential, but how many samples are necessary or enough? From what kind of perspectives can we try to estimate or/and determine how much data we should annotate? Or what kind of techniques can we use to predict the amount before we get the results of the first attempt?
I know that words can be separated into two types: closed class type and open class type. Should we consider that?
If such methods do not exist, how can we know if our model (mostly high variance) suffers from data shortage (knowing that data is never enough) or other reasons (like poor structure or bad training process or bad hyperparameters, etc.) after we get the results of our first version of the model and think the results are not satisfactory?