# How to do class balancing?

I am working with a really imbalanced dataset ($$\approx$$ 1% of positive cases) for a classification problem. I know that class balancing is an important step in this scenario.

I have two questions:

1. Considering that I don't want to put the 0/1 label, but just to order the record according to the output score (it is always a calibrated probability of being in the positive class), is it still a good idea to do class balancing or, considering the specific output required, it is useless?

Basically, I do not care about the cut-off point, but I just sort the record in order to identify the one with a higher probability of being positives.

2. Considering the really small percentage of positive cases, is it better to do over/under sampling? Is there any rule-of-thumb to decide the proportion of resampling?

With such a heavy imbalance and two classes (it seems) you could treat this as more of an outlier detection problem. You should read up on models and algorithms in that direction!

If you go forward with a traditional classification you need to balance the data set, consider methods such as SMOTE.

Depending on the size of your data I would generally recommend downsampling the majority class which avoids producing "synthetic" cases but advanced methods such as SMOTE basically take care of this decision for you.

Can you elaborate what you mean with your first question as well? A classification algorithm needs 0/1 labels and therefore the output score cannot be ordered in the way that you mention. Some classification algorithms put out a probability score instead of predicted label so if this is what you mean I can tell you that the imbalance will still be a problem.

• I have edited the question to provide more details about the second points. Let's consider that working on the output scores I can add the 0/1 label to each record using an ideal cut-off point (eg. 0.5), but it is useless for me since I just want to identify the top-ranked records in order to contact them (eg. to sell some products,...) Jul 20, 2020 at 9:46
• As said in this case this changes nothing about my answer and you'd need to deal with the imbalances. Jul 20, 2020 at 10:04
• About what @Fnguyen mentions, it sounds logic to look at this problem as a "novelty" detection problem due to the extreme imbalance; nevertheless, remembreing it would be a semi-supervised learning problem; two possible algorithms for that are scikit-learn.org/stable/modules/svm.html#svm-outlier-detection and scikit-learn.org/stable/modules/generated/…, both having an anomaly score with which you can tune your desired threshold Jul 31, 2020 at 15:27

Referring to a previous answer and a blog post (which I'm aware is not that relevant since the data is more balanced than yours), I think that your first approach should be without handling imbalance, and if you're happy with the results, no need to work towards balanced solutions.

As in many ML topics, the best way is to try, I recommend you to adapt the experiment in the blog post to your data.