In a binary supervised classification where classes 1 and 0 have different number of samples in training, it’s very common to find tutorials about adjusting class weights, over and under sampling for imbalanced data sets. In a situation where there are enough samples for both classes (e.g. not an anomaly detection), why would one adjust class weights or balance the training data if in the end you’ll have to adjust a threshold anyway?
1 Answer
If there are enough samples of both classes, I don't think it makes a lot of sense. I've been in kaggle competitions with very imbalanced datasets, like:
And none of the top solutions used any kind of treatment of the imbalance, as there were enough samples for both classes.
I've also done myself, several fraud-related models with high imbalance, and using no imbalance solution proved better than all the other options.