My target variable is whether an application is accepted or not. It is a highly imbalanced target with 98.5% of applications accepted. I am unclear about the concept of downsampling. If I were to downsample the applications, do I have to maintain the current ratio of accepted to rejected applications while lowering the total number of applications in the training data or can I change the ratio of accepted to rejected apps to say 50% accepted, 50% rejected? What are the benefits of doing either of these approaches? Help is welcome. Thanks!
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
Downsampling means you sample from the majority class (the 98.5%) to reduce the imbalance between majority and minority class. If you keep the ratio constant you simply reduce your number of trainings examples. This doesn't make sense. However, you don't have to sample down to a ratio of 50:50. If you have a ratio of 98:2, you can sample to 80:2 instead of 2:2.
The main goal of downsampling (and upsampling) is to increase the discriminative power between the two classes. Ideally, you would have a classifier that outputs a decision surface that is not simply binary (e.g. logistic regression (where you don't have to select a cut-off point of 0.5)) but gives you a continuous decision value. You can then order the data and set a decision threshold that gives you the best outcome.
Since downsampling (or upsampling) changes your training distribution from your true distribution, you only want to downsample (or upsample) so much that your classifier can start discriminating between the two classes. You then fine-tune the results by selecting an appropriate decision threshold. (Also, in my experience, upsampling is often a better choice over downsampling.)
-
$\begingroup$ Further question to that particular answer: What would be a rule-of-thumb for targeted ratio in case of downsampling, taking also into account whole sample size? $\endgroup$– PWillmsCommented Jul 8, 2020 at 17:10
$\begingroup$
$\endgroup$
Ideally, you should have the same distribution in the training data as in the test data, that is, it makes no sense to downsample for the reason you're talking.
However, when training your model you may want to assign larger weights to negative samples in order to optimise for f1_score rather than for accuracy.
$\begingroup$
$\endgroup$
Prefer to upsample the data to balance input classes (If your data is balanced you don't need to assign specific weight to any class specifically).
You can refer below link where I've given one small example to upscale input data.
https://datascience.stackexchange.com/a/40895/62202
