Suppose I want to train a binary model in order to predict the probability of who will buy a personal loan and in the dataset only 5 percent of the examples are people who marked as bought a personal loan. So, in this scenario maybe I can leverage downsampling or upsampling to balance the dataset, but if my dataset isn't big enough there may left very few examples or may be upsampling isn't appropriate. Then suppose I decided to use whole dataset, I partitioned it to the training and test sets in order to predict the probability of who won't buy a personal loan. Considering it's a binary model does it make sense to subtract this model's output probabilities from 1 and predicting who will buy a personal loan by using this result?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ When I have a problem like this, I have found it helpful to pursue a hierarchical approach. I first use decision trees to segment out a large portion of the larger class. Typically there will be leaves in the tree with virtually none of the smaller class. The other leaves can then comprise the modeling dataset, which will be more balanced. $\endgroup$– PaulCommented Jan 9, 2020 at 20:58
-
$\begingroup$ Sounds like a good approach. $\endgroup$– tkarahanCommented Jan 10, 2020 at 4:45
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Yes that's correct, but assuming that you follow the exact same methodology you will obtain exactly the same performance at the end, so there's no advantage.
Keep in mind that the problem with class imbalance is not that one class is harder to identify than the other, but rather that it's harder to properly separate the two classes.
[edit] It would be a different story when using one-class classification. I'm not sure if it makes sense in this case but maybe it could be something worth trying.
-
$\begingroup$ So in this way we push the decision boundary in order to get better results at predicting who will not buy, but also it means we get higher probabilities for misclassified examples and when we subtract their predicted probability from 1 it gives low probabilities and poor performance at predicting who will buy. $\endgroup$– tkarahanCommented Jan 9, 2020 at 18:11
-
$\begingroup$ @TolgaKarahan if the same proportion is used for the classes that wouldn't really "push the decision boundary", since the algorithm will try to minimize the error in both cases. If using different proportion (downsampling or upsampling) that will favor either precision or recall, but the performance obtained by focusing on one class will be the perfect mirror of the other class anyway. $\endgroup$– ErwanCommented Jan 9, 2020 at 18:24
-
$\begingroup$ PS: added a remark about the idea of one-class classification, in case that's of interest to you $\endgroup$– ErwanCommented Jan 9, 2020 at 18:27