1
$\begingroup$

I am using BERT to do multiclass text classification. The number of output classes I have to predict from is: 116 and there is high degree of class imbalance that I see.
We have the following kind of records available for each of the classes:
{'Class A': 975 number of records,
'Class B': 776 number of records,
'Class C': 533 number of records,
'Class D': 412 number of records,
'Class E': 302 number of records,
'Class F': 250 number of records,
'Class G': 207 number of records,
'Class H': 137 number of records,
'Class I': 96 number of records,
'Class J': 51 number of records,
'Class K': 28 number of records,
'Class L': 17 number of records,
'Class M': 7 number of records,
'Class N': 2 number of records}

So I have two questions here:
Question1: As we have around 116 output classes to predict from, does that affect the performance of BERT due to the high number of output classes?

Question2: My original data has the similar type of class distribution that I have illustrated above. So how does this affect the performance of BERT and if it affects how do we handle this to get proper output?

Looking forward to get answer from the talented community we have here.

Much thanks in advance.

$\endgroup$

1 Answer 1

1
$\begingroup$

Why does your distribution contains 14 classes? What about the 102 others?

Quick but generic answers:

  1. The number of classes always affect performance, because it's always easier to predict the correct class among a small number of possibilities than a big one. Even a random classifier has 50% chance to be correct with two classes, but only 1% chance with 100 classes. however performance also depend on the data, especially how easy it is to distinguish the classes.
  2. The imbalance is part of the problem. You shouldn't try to resample because it doesn't work with text data. you could consider data augmentation techniques, but I'm skeptical. Typically your very small classes like M,N don't have a representative sample, imho they should be discarded. It could be useful to start by training a classifier which deals only with a few large classes to see how well it works, and then to progressively add the other classes. Don't forget that performance might be better by ignoring some of the small classes.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.