I am working on a dataset with around 2000 records.

Around 80% records have their the categorical labels.

There are around 200 categories, some categories got more than 20 records; whereas others only have TWO....

Considering this is a text dataset, so I cannot do the oversampling for minority categories with techniques like what I could do for images.

I am using Fast AI which is based on PyTorch.

So what can I do for it?

  • 1
    $\begingroup$ You can perform text augmentation ( similar to image augmentation in CV ) for classes which have fewer samples. See this blog. $\endgroup$ Jun 28, 2021 at 4:13
  • $\begingroup$ Could you give a simplified example? $\endgroup$ Jun 28, 2021 at 9:52

1 Answer 1


Honestly it's unlikely that the model can handle these very small classes, because a few instances cannot be a sufficiently representative sample. Text implies high variations (there are many ways to express the same thing) so large samples are needed to cover the diversity of language.

Even larger classes might be difficult to learn, it depends how much they differ from each other in the data.

My advice:

  1. Start by implementing the system with only frequent enough classes, for instance remove the instances for classes which have less than 10 instances.
  2. Evaluate the system to make sure that it works well enough in this case.
  3. Only when/if everything works, try to deal with the harder case of small classes.

In some cases it can make sense to merge all the small classes into one single "other" class, but it depends on what the classes represent.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.