I have a data set that has some unbalanced categorical features. I would like to build a regression model to predict a label using machine learning (ML).

How do I handle data imbalances in categorical variables to avoid any bias in my ML regression model?


1 Answer 1


First, it depends on the number of samples and the degree of imbalance:

  • Small number of samples may cause slightly imbalanced features' categories to be trained very few times within the training stage
  • High imbalances can cause some categories to only appear in the training or in the validation/test stages

As a rule of thumb, I usually join the categories up to a point their distribution in the train and validation datasets are similar.

While this can lead to some bias, it avoids problems on handling categorical features when deploying in production or when testing in the test dataset.

Other strategy is to define a threshold (e.g. 5%) in which you join all the features categories which their distribution is below such threshold.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.