Disclaimer : This is a long question, please be patient. Thanks in advance
I am using bert-base-uncased for text-classification. I have 11 classes, and the classification is happening alright for most of the classes. But of these 11 classes there are three classes, say A, B and C. Where there are high misclassification errors. I wish to reduce the errors between these classes.
Current State of my model :
- Model used Hugging Face bert-base-uncased.
- Loss function : Weighted Cross Entropy where the weights represent the inverse of the fraction of each class in the data.
- The text data related to classes A, B and C are not unbalanced and are roughly comparable to the most populous class
My Questions :
- Can anyone say why is this occuring?
- I am thinking of using some-other loss function specifically for these three classes, say soft-F1 from torchmetrics. The idea is that nn.Cross_Entropy() will be used for all classes and apart from that I will use soft-F1 when the true_label belongs to these three classes.
Thus the final loss function will be
loss = frac * $nn.Cross_entropy() + (1-frac) * torchmetrics.soft_f1(if true_label in [A, B, C]), where
frac is an hyper-parameter.
Will this approach work, or should I use something else ?