0
$\begingroup$

I am working on a binary classifier using LightGBM. I try to see the results of the classifiers when changing the costs of false positives and false negatives, still working on the same training and validating datasets. As I want to have probabilities as a result of my modelling, I use isotonic regression as a final part of the pipeline.

Applying exactly the same methodology and code, but only changing those variables of customized objective function, I can see that most of the classifiers are perfectly calibrated, but just a few are not - all of those few are below the 'perfectly calibrated' line.

Why has that happened? How come the calibration cannot be perfect - overall and in this scenario?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.