# How to predict probabilities in xgboost using R?

The below predict function is giving -ve values as well so it cannot be probabilities.

param <- list(max.depth = 5, eta = 0.01,  objective="binary:logistic",subsample=0.9)
bst <- xgboost(param, data = x_mat, label = y_mat,nround = 3000)

pred_s <- predict(bst, x_mat_s2)


I google & tried pred_s <- predict(bst, x_mat_s2,type="response") but it didn't work.

Question

• Doesn't it output probabilities by default with the settings you used? I mean: have you examined pred_s and you are certain those are not probabilities?
– kpb
Sep 8, 2015 at 11:58
• No its having negative values. Probability should vary between 0 to 1. Sep 8, 2015 at 12:05
• I don't see any obvious issues. (Although, I'm more familiar with the python wrapper). Have you tried adding outputmargin=F to the predict function? If somehow the outputmargin is set to T, it will return the value before the logistic transformation. Sep 10, 2015 at 17:24
• For Python, you can copy predict_proba implementation from sklearn API: github.com/dmlc/xgboost/blob/master/python-package/xgboost/… Jan 19, 2018 at 15:39

Just use predict_proba instead of predict. You can leave the objective as binary:logistic.

• If this were Python and not R, then this answer might be sensible. Wrong language. Aug 15, 2016 at 18:30
• oops! thanks @B_Miner. I'm not deleting this answer as it might be helpful for others that will make the same mistake and think we're talking about python.. Aug 16, 2016 at 8:49
• For me this does not do the trick datascience.stackexchange.com/questions/14527/… Nov 8, 2016 at 7:59
• xgboost does not have a predict_proba function Aug 24, 2017 at 19:26
• XGBoost Classifier does has a predict_proba option xgboost.readthedocs.io/en/latest/python/python_api.html Dec 24, 2019 at 19:07

Know I'm a bit late, but to get probabilities from xgboost you should specify multi:softmax objective like this:

xgboost(param, data = x_mat, label = y_mat,nround = 3000, objective='multi:softprob')


From the ?xgb.train:

multi:softprob same as softmax, but output a vector of ndata * nclass, which can be further reshaped to ndata, nclass matrix. The result contains predicted probabilities of each data point belonging to each class.

• Thanks. How is this loss function different from binary:logistic for binary classification? Nov 12, 2015 at 7:30
• It's just a generalization of logistic function for multi-class case, should be no significant difference. Nov 12, 2015 at 7:39
• I used it and it worked well, but I had an issue. I'm facing a warning "Parameters: { scale_pos_weight } might not be used.", which comes form the fact I use multi:softprob (source). Why my unbalanced data, it seems that this param is really important (if I go for a classic binary classification), and currently, when I change it, it has no impact because it's not taken into account. Aug 13, 2020 at 8:06

After the prediction:

pred_s <- predict(bst, x_mat_s2)


You can get the probability by:

pred_s$data  If this is a binary classification then pred_s$data includes prob.0, prob.1, response.

So you can get prob.1 by:

pred_s$$data$$prob.1