0
$\begingroup$

The dataset that I have is some text data consisting of path names. I am using TF-IDF vectorizer and decision trees. The classes in my dataset are severely imbalanced. There are a few big classes with a number of samples more than 500 and some other minor classes with a number of samples less than 100. Some are even smaller (less than 20). This is real data collected, so the chance where the model seeing minor class in actual implementation is rare as well. The problem I am having right now is, the model predicted the minor class as the major class most of the time, causing my accuracy to be at around 45% all the time. If I resample the data, I think the accuracy will be even worse as the model's ability to learn the major class is reduced.

So I would like to ask if I should consider resample my data or anyone have any suggestion on how to improve the accuracy of my model? Any help is appreciated.

$\endgroup$
2
0
$\begingroup$

Resampling is generally a good idea when dealing with heavily imbalanced datasets. I would recommend using smote which can either resample or undersample depending on your task (if your dataset is small I would say to use resampling). Another thing you could try is using class weights when training. Models like lightgbm and neural nets can use weighted training losses to force models to pay attention to underpopulated classes. I have personally dealt with such cases and these methods definitely work.

$\endgroup$
1
  • $\begingroup$ Please see my comment. $\endgroup$
    – Dave
    Jul 15 at 1:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.