We have an optimization problem on hand that is the following.
Let's say we have 10 different treatments that we might offer, all of them are equally good for us, but people have different propensity to take those treatments.
Before we've built the model the rule was to just randomly choose one of the treatments, offer it to a person, and if the person takes it - yaay, if not - so be it.
Based on this historic data we've created a binary classifier that predicts a person's propensity to agree to each treatment type and the idea is to offer a person a treatment that has a max propensity score from the model.
The model built shows great GINI and AUC numbers, but what we struggle with is a more business-related metric that we could use to evaluate the model's performance.
I'm sure some of you had been in this situation before. What metrics did you use?