First, it depends on the number of samples and the degree of imbalance:
- Small number of samples may cause slightly imbalanced features' categories to be trained very few times within the training stage
- High imbalances can cause some categories to only appear in the training or in the validation/test stages
As a rule of thumb, I usually join the categories up to a point their distribution in the train and validation datasets are similar.
While this can lead to some bias, it avoids problems on handling categorical features when deploying in production or when testing in the test dataset.
Other strategy is to define a threshold (e.g. 5%) in which you join all the features categories which their distribution is below such threshold.