Questions tagged [probability-calibration]

The tag has no usage guidance.

10 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
346 views

How to explain a Calibration Plot for many models?

I have a heavy imbalanced dataset with a classification problem. I try to plot the Calibration Curve from the sklearn.calibration package. In specific, I try the ...
2
votes
1answer
35 views

why does my calibration curve for platts and isotonic have less points than my uncalibrated model?

i train a model using grid search then i use the best parameters from this to define my chosen model. ...
2
votes
0answers
84 views

Platt Scaling vs Isotonic Regression for reliability curve

I am learning classifier probability calibrations and have calibrated an eleastic net model using both Platt scaling and isotonic regression. As you can see in the attached image Platt scaling (on the ...
1
vote
0answers
36 views

xgboost calibration kde plots (isotonic) not smooth

i am training my xgboost model on an imbalanced binary classification problem. It is important to me to have well calibrated probabilities so i have chosen to optimize the brier score. I then plot the ...
1
vote
0answers
29 views

calibrating classifier probabilities for unbalanced data when class ratios are unknown

I've built a binary classification convolutional neutral network, trained on simulated data with equal numbers of simulations for each class. I've obtained good results for a validation set with equal ...
1
vote
0answers
19 views

How to evaluate a model based on the effect of one important variable?

We want a well-calibrated classifier that tells us the probability of an event. The model has multiple inputs, but we are interested in how the probability of an event changes as we vary one input. ...
1
vote
2answers
40 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
1
vote
0answers
92 views

Probability Calibration : For 2D image data, how to use the calibration?

I have a model which takes 2D input data and does multi class classification in keras. I would like to plot the probability calibration curve. However, using the scikit function, it returns an error ...
0
votes
0answers
16 views

using logistic:binary as objective in xgboost

i am using 'binary:logisitic' as my objective function in xgboost classifier. Therefore when i predict probabilities i.e: model.predict_proba(x_test)[:, 1] will ...
0
votes
0answers
28 views

Imbalanced text classification by oversampling: correction of class predicted probability by prior probability

My dataset has 3 class and 900 examples for training. Class distribution is 255, 185, and 460. I found that if I oversample (random) the training data then I have to correct/calibrate the predicted ...