# Questions tagged [probability-calibration]

The tag has no usage guidance.

13 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0answers
383 views

### How to explain a Calibration Plot for many models?

I have a heavy imbalanced dataset with a classification problem. I try to plot the Calibration Curve from the sklearn.calibration package. In specific, I try the ...
0answers
134 views

### Platt Scaling vs Isotonic Regression for reliability curve

I am learning classifier probability calibrations and have calibrated an eleastic net model using both Platt scaling and isotonic regression. As you can see in the attached image Platt scaling (on the ...
0answers
15 views

### Account for imbalanced data in a Neural Network using prior distribution

I have a dataset with 4 classes, say their distribution in the training-set is $P_{prior}(C1) = 60\%$ $P_{prior}(C2) = 25\%$ $P_{prior}(C3) = 10\%$ $P_{prior}(C4) = 5\%$ After training a Neural ...
0answers
73 views

### xgboost calibration kde plots (isotonic) not smooth

i am training my xgboost model on an imbalanced binary classification problem. It is important to me to have well calibrated probabilities so i have chosen to optimize the brier score. I then plot the ...
0answers
37 views

### calibrating classifier probabilities for unbalanced data when class ratios are unknown

I've built a binary classification convolutional neutral network, trained on simulated data with equal numbers of simulations for each class. I've obtained good results for a validation set with equal ...
0answers
72 views

### Imbalanced text classification by oversampling: correction of class predicted probability by prior probability

My dataset has 3 class and 900 examples for training. Class distribution is 255, 185, and 460. I found that if I oversample (random) the training data then I have to correct/calibrate the predicted ...
0answers
19 views

### How to evaluate a model based on the effect of one important variable?

We want a well-calibrated classifier that tells us the probability of an event. The model has multiple inputs, but we are interested in how the probability of an event changes as we vary one input. ...
2answers
49 views

### Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
0answers
123 views

### Probability Calibration : For 2D image data, how to use the calibration?

I have a model which takes 2D input data and does multi class classification in keras. I would like to plot the probability calibration curve. However, using the scikit function, it returns an error ...
0answers
33 views

### Implementing Smoothed Isotonic Regression

In the paper here the authors suggest a new way of calibrating classifiers, called Smoothed Isotonic Regression (Algorithm 1). As I follow the algorithm along, I noticed a problem in lines 19-20: ...
0answers
18 views

### Calibration of a few binary classifiers is not perfect - why?

I am working on a binary classifier using LightGBM. I try to see the results of the classifiers when changing the costs of false positives and false negatives, still working on the same training and ...
0answers
32 views

### Calibration curve motivation

I struggle to understand the mathematical motivation for the binary classification model calibration curve. Why do we assume that the predicted probabilities should be consistent with the proportion ...
0answers
25 views

### How to ouput buckets of probabilities?

I am dealing with an unbalanced binary classification problem. The problem is so unbalanced (2:98) and hard to predict that I am interested in probability of the positive outcome instead of trying to ...