0
$\begingroup$

I have 5 classes. I made a XGBoost Classification model and used model.predict(test) to predict the classes of test dataset. Out of all those values predicted by my model, I would like to know only those values that my model is more than 95% sure that the predicted value is correct. I mean, I would only like those predictions that my model is very confident of predicting. How do I find those predictions?

$\endgroup$
1
  • $\begingroup$ Use probabilistic predictions. I don't remember the inner workings of XGBoost, but methods like neural networks give probability outputs that can be converted to class membership using a cutoff threshold (often $0.5$ as a software default), but you can do whatever you want with the probability outputs. This has to do with something called a proper scoring rule that evaluates the probability predictions rather than the threshold-based classes. $\endgroup$
    – Dave
    Aug 13, 2021 at 19:46

1 Answer 1

1
$\begingroup$

Have a look at the predict_proba method of the XGBClassifier class which will give you the probabilities for each class instead of just the predicted class. You can then use these probabilities to only select the class with the highest probability if the probability is above the treshold you want to set (in this case 0.95).

$\endgroup$
3
  • 2
    $\begingroup$ Note that with five classes, it’s entirely possible that none of them will have a probability anywhere close to $0.95$. (This is a feature, not a bug. If the case is vague, the model should be uncertain.) $\endgroup$
    – Dave
    Aug 13, 2021 at 20:59
  • $\begingroup$ I used predict proba. It gave an array as results. One of the value in that array is 0.85, does this means my model is 85% sure of predicting it? $\endgroup$ Aug 14, 2021 at 4:07
  • $\begingroup$ A value 0.85 would mean that the model thinks that the input has a 85% chance of being in that specific class. $\endgroup$
    – Oxbowerce
    Aug 14, 2021 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.