2
$\begingroup$

I'm dealing with a multi-class classification problem with around 30 categories.

This problem has a severe class imbalance:

  • Around 300 examples for the least common class.
  • Around 100k examples for the most common class.

I don't want the classification model to be dummy and predict the most common class for most of the examples, for this reason, I'm using class_weight='balanced' in my LogisticRegression from sklearn. However, in this case, the classes that the algorithm predicts are mostly the less frequent ones. I understand the model overfits them somehow, as it assigns every sample from these class a very high weight.

On the other hand, if I don't apply the class weights, the model predicts the most common categories.

Is there a way to solve this? Is there a way to ensure the model predicts approximately the same proportion of samples for each category?

$\endgroup$
1
$\begingroup$

There are probably many different strategies but it's a difficult problem when the imbalance is as severe as it is here.

Without any correction the model is likely to ignore the smallest classes, as you noticed. However forcing the class weight as if the data is balanced is certainly too strong a correction. A middle ground would be to resample the training set instances yourself before fitting the model: by trying different ways to undersample the large classes and/or oversample the small classes you should be able to find an optimal tradeoff between the two extremes (use a separate validation set to determine the optimal combination).

Is there a way to ensure the model predicts approximately the same proportion of samples for each category?

Maybe I misunderstood but this looks like a bad idea: if the true proportions are not equal then the model shouldn't predict equal proportions either. The ideal scenario is for the model to predict the correct label every time, which implies predicting the true proportion for every class.

It might also be useful to analyze the performance in simpler configurations, e.g. by picking a few "average size" classes and observing how well the classifier discriminates between them only. The harder it is for a classifier to predict correctly, the more it relies on basic class proportion since it doesn't know any better.

$\endgroup$
1
  • 1
    $\begingroup$ Another approach would be to manually adjust the class weights (even treat them as hyper-parameters and search). This could also improve the outcome $\endgroup$ – Nikos M. May 4 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.