I have tried a 2 different versions of a gbm in a multinomial classification problem. The second model results in better confusion matrix but in worse Log Loss value (at the test sample). How is that possible.

Further are the results of the two models.

enter image description here

I thought that it could be because the Class A is much more oversampled and the small decrease of that class could lead to such a deterioration of hte logloss?

Any ideas? Thank you


Log loss is a measure of confidence of the model in its predictions. Lower log loss implies higher confidence and vice-versa. Better confusion matrix with worse log-loss implies that few of the misclassifications, but with higher probability score. Since the accuracy is calculated with a fixed probability threshold, you might not observe this directly. Plot the ROC curves for both the models with is based on varying the decision threshold. This gives you the idea about which model is better.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.