Questions tagged [probability-calibration]
The probability-calibration tag has no usage guidance.
15
questions
0
votes
2answers
58 views
Compare scores of models
We got several models with predictions. How can we compare scores of different models with each other?
We assume that we got xgboost models and scores distribution can be different for each model, so ...
1
vote
1answer
39 views
Probability calibration is worsening my model performance
I'm using RandomForest and XGBoost for binary classification, and my task is to predict probabilities for each class. Since tree-based models are bad with outputting usable probabilities, i imported ...
2
votes
1answer
47 views
Predict_proba on a binary classification problem
I have a binary classification task on my hands, i have a bunch of people that i need to classify as being ones or zeros and then use predict_proba to estimate how confident my prediction was on the ...
1
vote
0answers
18 views
How to evaluate a model based on the effect of one important variable?
We want a well-calibrated classifier that tells us the probability of an event. The model has multiple inputs, but we are interested in how the probability of an event changes as we vary one input. ...
1
vote
2answers
79 views
convert predict_proba results using class_weight in training
As my dataset is unbalanced(class 1: 5%, class 0: 95%) I have used class_weight="balanced" parameter to train a random forest classification model. In this way I penalize the misclassification of a ...
1
vote
1answer
27 views
Determining threshold in an area with very few samples of positive label
I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
1
vote
0answers
41 views
Probability Calibration : For 2D image data, how to use the calibration?
I have a model which takes 2D input data and does multi class classification in keras. I would like to plot the probability calibration curve. However, using the scikit function, it returns an error ...
2
votes
0answers
149 views
How to explain a Calibration Plot for many models?
I have a heavy imbalanced dataset with a classification problem. I try to plot the Calibration Curve from the sklearn.calibration package. In specific, I try the ...
1
vote
1answer
41 views
Does PMML support probability calibration?
As I need to port a decision tree model from Python to Java, I would like to know whether PMML (Predictive Model Markup Language) supports probability calibration.
1
vote
1answer
193 views
Calibrate the predicted class probability to make it represent a true probability?
Let's say that we have a simple binary classification model (a neural network -- NN) for classifying input images as "dog" ($y=1$) or "not dog" ($y=0$). Let's assume that the NN has one "sigmoid ...
0
votes
2answers
134 views
Do I need to correct predict_proba by training fraction?
Many algorithms provide a predict_proba function indicating probability of a case to belong to that class (e.g. https://scikit-learn.org/stable/modules/generated/...
6
votes
2answers
760 views
Probability Calibration : role of hidden layer in Neural Network
I try a simple Neural Network (Logistic Regression) to play with Keras.
In input I have 5,000 features (output of a simple tf-idf vectorizer) and in the output layer I just use a random uniform ...
4
votes
1answer
2k views
Are the raw probabilities obtained from XGBoost, representative of the true underlying probabilties?
1) Is it feasible to use the raw probabilities obtained from XGBoost, e.g. probabilities obtained within the range of 0.4-0.5, as a true representation of approximately 40%-50% chance of an event ...
7
votes
2answers
455 views
XGBoost outputs tend towards the extremes
I am currently using XGBoost for risk prediction, it seems to be doing a good job in the binary classification department but the probability outputs are way off.
i.e., Changing the value of a ...
4
votes
0answers
93 views
which loss function (if any) optimizes the calibration graph
The calibration graph is the predicted versus actual probability(see http://scikit-learn.org/stable/modules/generated/sklearn.calibration.calibration_curve.html). Is it possible to optimize the ...
