Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
we are creating models that aim to filter new leads from our current customer base.
We started to create propensity models that calculate a percentage for each customer for a certain product group. I was wondering what are best practices to benchmark such a model and to show the success of a propensity model in advance.
Thx for your replies!
I'm assuming that the "propensity model" predicts a customer's likelihood to take some desired action.
If it's possible at your organization, then a good way to measure the model's effectiveness is an A/B test. Select a set of leads to act as a control group using a baseline selection method. Ideally, the baseline method would be whatever is currently in-place at your organization. Then allow your model to select a set of leads as the experimental group. Compare conversion rates (or whatever else you care about) between the control group and the experimental group.
If A/B testing isn't viable, then you can usually evaluate your model using the traditional techniques. You can split your customer base into train/test sets. Train your model on the training set, then made propensity predictions for the test set. Compare the propensity estimates with actual conversion rates.
