I was doing churn prediction for a company. I've got the following results by applying 3 classifier.

Model Accuracy AUC
Logistic Regression 0.671 0.736
Decision Tree (pruned) 0.681 0.665
Decision Tree unpruned 0.623 0.627

Now, I want to know two things:

  • which model has a better accuracy for a cutoff of 0.9?

As the logistic regression has highest AUC so, in my opinion, Logistic Regression is better

  • Which model is the best in terms of ranking the predictions according to their probability of leaving

Can anyone explain how I can interpret them?

  • $\begingroup$ How do you see AUC as measuring accuracy with a particular cutoff threshold? $\endgroup$
    – Dave
    Commented Oct 18, 2021 at 3:52

1 Answer 1

  • The accuracy is likely to go down if you change the cutoff point to 0.9, since any model tries to separate the classes so that the probability of the correct class is higher than 0.5. But the only way to know would be to actually do the experiment (I assume that the results that you show are obtained with the default cutoff).
  • AUC is a complex measure for a soft classifier, i.e. it doesn't use any cutoff point but provides a performance value across cutoff points. Importantly, the AUC score considers one class as positive, like the precision, recall and F1-score measures. Btw I would suggest looking at these scores instead, which are more precise than accuracy and more easily interpretable than AUC.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.