1
$\begingroup$

I have the following evaluation metrics on the test set, after running 6 models for a binary classification problem:

  accuracy logloss   AUC
1   19%      0.45   0.54
2   67%      0.62   0.67
3   66%      0.63   0.68
4   67%      0.62   0.66
5   63%      0.61   0.66
6   65%      0.68   0.42

I have the following questions:

  • How can model 1 be the best in terms of logloss (the logloss is the closest to 0) since it performs the worst (in terms of accuracy). What does that mean ?
  • How come does model 6 have lower AUC score than e.g. model 5, when model 6 has better accuracy. What does that mean ?
  • Is there a way to say which of these 6 models is the best ?
$\endgroup$
2
  • $\begingroup$ What's the ratio of the classes in your dataset? $\endgroup$
    – Ben Reiniger
    Oct 30, 2019 at 2:40
  • $\begingroup$ the ratio is around 1/5 $\endgroup$
    – quant
    Oct 30, 2019 at 7:44

1 Answer 1

-1
$\begingroup$

I will start with your last question as the answer explains the other questions. In general, this is a hard question. That is exactly why there are different metrics available. Depending on your problem and what you want to achieve, you should choose the metric that measures this best.

The differences in metrics can be found all over the internet. Others have explained it much better than I can.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.