I am learning classifier probability calibrations and have calibrated an eleastic net model using both Platt scaling and isotonic regression. As you can see in the attached image Platt scaling (on the bottom) better approximates the diagonal line compared to isotonic regression (top), however I noticed I am losing information with any predictions where the predicted probability <0.4, I have seen this happen in uncalibrated plots as well. Therefore I am wondering which calibration method I should be using. Furthermore, generally what about the model/data causes the curve to be cut like this and are there things I can do to address this during modeling?

Many thanks!

Update: per suggestion included a plot of the uncalibrated probabilities for comparison purposes. uncalibrated

top is after isotonic regression, bottom is post-platt scaling

  • $\begingroup$ Could you add the plot before calibration? Might be helpful to answer your question. $\endgroup$ – oW_ Apr 22 at 14:35
  • $\begingroup$ edit: original post now includes a plot of the uncalibrated probabilities. Thank you oW_ $\endgroup$ – yl637 Apr 23 at 6:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.