I am trying to train my ML model to classify 13 intents. Prior to training the model, the training data was sufficiently accurate. Here are the imbalanced training data sizes per class (class_num: training_size):
1:31, 2:340, 3:1368, 4:20496, 5:4160, 6:19208, 7:1860, 8: 3751, 9: 223146, 10:19, 11:5, 12:37, 13:27
Overall, all the phrases attached to all the classes from 1-11 are performing fine. However, when it came to classifying 12 and 13, none of the phrases are predicted correctly at all. They instead classify mostly as 4 and sometimes 3. Upon further inspection, I noticed that class #4 contains a lot of phrases with the same keyword that is prevalent in 12 and 13. However, 12 and 13 are the only classes that use this keyword in conjunction with the verb "create"/"add"/"make". How would I go about re-training my CNN model to properly classify these classes? The reason for the imbalance is that some intents have more phrase options as well as entities that can go into the placeholders. So the data augmentation process is able to place these entities into the placeholders and generate a higher training set for those intents. However, classes 12 and 13 are simpler phrases with no placeholders for entities.
I tried increasing the epoch count, increasing the weights on these 2 intents to be 50 times higher, and downsampled for labels 3,4, and 9. None of them were able to generate even 1 accurate prediction for 12 and 13. My next approach would be to oversample the training data (i.e. copy-pasting the rows several times in my training data) for these intents so that they have a few hundred samples. However, I feel like this is a very crude workaround and was wondering if anyone has a more elegant solution.