We want a well-calibrated classifier that tells us the probability of an event. The model has multiple inputs, but we are interested in how the probability of an event changes as we vary one input. This is the primary use of the model so it is very important.

Ex. What is the probability that you buy my sandwich?

Inputs: meat type, cheese type, sandwich weight, price

The real question of interest is, how does the probability of buying the sandwich change with the price?

I am a sandwich shop using this model to set my sandwich prices, so I really mostly care about how you respond to changes in price. But I still need the other variables because you have sandwich preferences that impact how much you are willing to pay.

Suppose we have several candidate models with similar performance in AUC and calibration, but each one shows a different effect of price on probability. How do we select which model to use?

For the same sandwich, model 1 vs model 2 give different probability of buying the sandwich as you vary the price input: enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.