I'm trying to predict response of customers on a marketing campaign. As of now, I have data from one Marketing campaign and rfm data of my customers.

Some proportion of customers say 60%, received adverts. Roughly 10% responded

From the response data, i.e. whether purchased or not, on this marketing campaign I built a random forest using scikit-learn.

The model performs on a hold out set really well. but the most influential variable is the boolean: CustomerHasBeenAdvertised

I want to use this model, to select customers for a future marketing campaign. To obtain the "purchasing probabilities" of the customers under the condition of similar advertisement, I set the variable CustomerHasBeenAdvertised to 1.

However, on a data set with this side condition, all the forecasts are above 0.5.

Is this extra ordinary high value due to the variable importance? Or are there other explanations?

Is setting the variable CustomerHasBeenAdvertised to 1 the false approach?

If so, how could one handle the case: Customer bought without beeing advertised?

Should one simply neglect the information whether advertisement took place or not?

Thanks in advance


2 Answers 2


It depends what you want to do. If you want a model that will predict who to target/not-target for a marketing campaign, then you want to train a model using only the people who were marketed to, which defines your two classes "responders" and "nonresponders". Given an unseen customer, your classifier will then determine whether they are likely to respond if you send them a directed marketing advertisement. Take a look at uplift marketing.

  • $\begingroup$ Actually, I did this approach, but the results weren't as good as i expected. The ultimate goal is to build a chain of models: the first model provides the purchase probability under the condition that the target obtained adverts or not. The second model predicts the net value a customer purchases for, if he does purchase (under the condition of adverts or not). "Multiplication" should yield an expected net profit of the customer under the condition of adverts or not. With this KPI the amount spent on (print) marketing campaigns could be optimized. Thanks for the reply $\endgroup$ May 3, 2016 at 20:19

If the CustomerHasBeenAdvertised variable corresponds to whether the customer received an ad from your campaign, then I think the best approach would be to exclude all records from your training set that do not have CustomerHasBeenAdvertised set to 1 and then eliminate that variable as a predictor. This could still result in most forecasts being above 0.5, but that's not necessarily a bad thing.

One common approach to dealing with imbalanced class problems (ex: few people responding to an ad or few people going bankrupt on a loan) is to separate your predictions into deciles, and then only take action on the top few deciles of interest (in your case customers most likely to make a purchase). This means that the relative / rank-order value of the prediction is more important than the nominal value of the prediction. Normally this is sufficient because you probably don't want to advertise to all customers as that would be cost prohibitive.

  • $\begingroup$ Yes, the variables acts like its name suggests it. Actually, I built a model only taking the advertised customers into account. However, the model did not work well on a hold out set. The "score" on the training data was roughly 1, whereas the score on the hold out data was merely 0.9 indicating strong overfitting. Adding the customers without advertisement put the test score on 0.99. The scikit learn random forest classifier has a built in method to account for class imbalances. Nevertheless, I'll conduct further tests tomorrow. Nevertheless, thanks for the reply $\endgroup$ May 3, 2016 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.