1
$\begingroup$

we are creating models that aim to filter new leads from our current customer base.

We started to create propensity models that calculate a percentage for each customer for a certain product group. I was wondering what are best practices to benchmark such a model and to show the success of a propensity model in advance.

Thx for your replies!

$\endgroup$

1 Answer 1

2
$\begingroup$

I'm assuming that the "propensity model" predicts a customer's likelihood to take some desired action.

If it's possible at your organization, then a good way to measure the model's effectiveness is an A/B test. Select a set of leads to act as a control group using a baseline selection method. Ideally, the baseline method would be whatever is currently in-place at your organization. Then allow your model to select a set of leads as the experimental group. Compare conversion rates (or whatever else you care about) between the control group and the experimental group.

If A/B testing isn't viable, then you can usually evaluate your model using the traditional techniques. You can split your customer base into train/test sets. Train your model on the training set, then made propensity predictions for the test set. Compare the propensity estimates with actual conversion rates.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.