# How to predict probabilities in xgboost?

The below predict function is giving -ve values as well so it cannot be probabilities.

param <- list(max.depth = 5, eta = 0.01,  objective="binary:logistic",subsample=0.9)
bst <- xgboost(param, data = x_mat, label = y_mat,nround = 3000)

pred_s <- predict(bst, x_mat_s2)


I google & tried pred_s <- predict(bst, x_mat_s2,type="response") but it didn't work.

Question

• Doesn't it output probabilities by default with the settings you used? I mean: have you examined pred_s and you are certain those are not probabilities? – kpb Sep 8 '15 at 11:58
• No its having negative values. Probability should vary between 0 to 1. – GeorgeOfTheRF Sep 8 '15 at 12:05
• I don't see any obvious issues. (Although, I'm more familiar with the python wrapper). Have you tried adding outputmargin=F to the predict function? If somehow the outputmargin is set to T, it will return the value before the logistic transformation. – inversion Sep 10 '15 at 17:24
• For Python, you can copy predict_proba implementation from sklearn API: github.com/dmlc/xgboost/blob/master/python-package/xgboost/… – Anton Tarasenko Jan 19 '18 at 15:39

Just use predict_proba instead of predict. You can leave the objective as binary:logistic.

• If this were Python and not R, then this answer might be sensible. Wrong language. – B_Miner Aug 15 '16 at 18:30
• oops! thanks @B_Miner. I'm not deleting this answer as it might be helpful for others that will make the same mistake and think we're talking about python.. – ihadanny Aug 16 '16 at 8:49
• For me this does not do the trick datascience.stackexchange.com/questions/14527/… – Georg Heiler Nov 8 '16 at 7:59
• xgboost does not have a predict_proba function – Ashoka Lella Aug 24 '17 at 19:26
• XGBoost Classifier does has a predict_proba option xgboost.readthedocs.io/en/latest/python/python_api.html – Paul Bendevis Dec 24 '19 at 19:07

Know I'm a bit late, but to get probabilities from xgboost you should specify multi:softmax objective like this:

xgboost(param, data = x_mat, label = y_mat,nround = 3000, objective='multi:softprob')


From the ?xgb.train:

multi:softprob same as softmax, but output a vector of ndata * nclass, which can be further reshaped to ndata, nclass matrix. The result contains predicted probabilities of each data point belonging to each class.

• Thanks. How is this loss function different from binary:logistic for binary classification? – GeorgeOfTheRF Nov 12 '15 at 7:30
• It's just a generalization of logistic function for multi-class case, should be no significant difference. – cyberj0g Nov 12 '15 at 7:39
• I used it and it worked well, but I had an issue. I'm facing a warning "Parameters: { scale_pos_weight } might not be used.", which comes form the fact I use multi:softprob (source). Why my unbalanced data, it seems that this param is really important (if I go for a classic binary classification), and currently, when I change it, it has no impact because it's not taken into account. – BeamsAdept Aug 13 '20 at 8:06

after the prediction

pred_s <- predict(bst, x_mat_s2)


you can get the probability by

pred_s$data  If this is a binary classification then pred_s$data includes prob.0, prob.1, response.

So you can get prob.1 by

pred_s$$data$$prob.1