0
$\begingroup$

I am building a model to predict if a customer will use a coupon or not for a given campaign. I am using logistic regression for this model. I took 5 previous campaigns and generally for each campaign conversion rate is around 10%. Thus, to handle this imbalanced data set and to capture more info I took a stratified sample from this data(whole 5 campaigns) such that there are 50% of sample with negatives and 50% positives. Thus, I am oversampling my positives.

My doubt is if I use logistic regression where it estimates coefficients using maximum log likelihood. Will this oversampling will generate bias results?

Also, I think this oversampling won't create any problem with random forest?

| improve this question | |
$\endgroup$
  • $\begingroup$ did you actually oversample the minority or downsampled the majority? My understanding is that you did the second. In any case these techniques are to balance imbalanced datasets and there is bias introduced no matter the classifier used, however it is hoped this is reverse bias to balance the bias from lack of minority samples $\endgroup$ – Nikos M. Jul 25 at 15:46
  • $\begingroup$ I have oversampled the minority by taking 50% of positive observations rather than 10% for a training sample. My doubt is around how the behavior of logistic regression changes when we introduce such bias especially when the sample is not exactly like population. I think random forest does not suffer from it severely as the additional observations are not synthetic data but true observations. Thus, the path to the leaf node should remain valid. $\endgroup$ – Mayank Mittal Jul 25 at 16:09
  • 1
    $\begingroup$ this post provides some rough theoretical points for your question $\endgroup$ – Nikos M. Jul 26 at 8:40
1
$\begingroup$

The effect will be increasing the intercept. I don't recommend doing oversampling unless any other solution doesn't work. Besides, 10% is not such a big imbalance.

I've been in kaggle competitions with way more imbalance where no imbalance solutions were adopted, logistic regression and random forest work quite well without the need of these.

Edit

After @Ben Reininger input, here's a theoretical justification on how does the intercept change.

Also, a quick experiment showcasing that over-sampling doesn't help improve a metric like AUC, and that it indeed increases the intercept of the logistic regression model.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I would rather have this answer as comment, as it is opinion/experience-based rather than theoreticaly based, which I think is what the OP is asking.. IMO $\endgroup$ – Nikos M. Jul 26 at 8:07
  • $\begingroup$ @NikosM. Couldn't agree more. Of course, I have to over sample the data based on my problem. As, with 10% logistic was running fine but random forest was predicting probabilities in short range. I want to understand is there any theoretical answer how oversampling affect logistic regression. $\endgroup$ – Mayank Mittal Jul 26 at 8:35
  • 1
    $\begingroup$ A reference for the first sentence's claim: stats.stackexchange.com/a/68726/232706 $\endgroup$ – Ben Reiniger Jul 26 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.