The below predict function is giving -ve values as well so it cannot be probabilities.

param <- list(max.depth = 5, eta = 0.01,  objective="binary:logistic",subsample=0.9)
bst <- xgboost(param, data = x_mat, label = y_mat,nround = 3000)

pred_s <- predict(bst, x_mat_s2)

I google & tried pred_s <- predict(bst, x_mat_s2,type="response") but it didn't work.


How to predict probabilities instead?

  • $\begingroup$ Doesn't it output probabilities by default with the settings you used? I mean: have you examined pred_s and you are certain those are not probabilities? $\endgroup$ – kpb Sep 8 '15 at 11:58
  • $\begingroup$ No its having negative values. Probability should vary between 0 to 1. $\endgroup$ – GeorgeOfTheRF Sep 8 '15 at 12:05
  • $\begingroup$ I don't see any obvious issues. (Although, I'm more familiar with the python wrapper). Have you tried adding outputmargin=F to the predict function? If somehow the outputmargin is set to T, it will return the value before the logistic transformation. $\endgroup$ – inversion Sep 10 '15 at 17:24
  • 1
    $\begingroup$ For Python, you can copy predict_proba implementation from sklearn API: github.com/dmlc/xgboost/blob/master/python-package/xgboost/… $\endgroup$ – Anton Tarasenko Jan 19 '18 at 15:39

Just use predict_proba instead of predict. You can leave the objective as binary:logistic.

  • 4
    $\begingroup$ If this were Python and not R, then this answer might be sensible. Wrong language. $\endgroup$ – B_Miner Aug 15 '16 at 18:30
  • 2
    $\begingroup$ oops! thanks @B_Miner. I'm not deleting this answer as it might be helpful for others that will make the same mistake and think we're talking about python.. $\endgroup$ – ihadanny Aug 16 '16 at 8:49
  • $\begingroup$ For me this does not do the trick datascience.stackexchange.com/questions/14527/… $\endgroup$ – Georg Heiler Nov 8 '16 at 7:59
  • 6
    $\begingroup$ xgboost does not have a predict_proba function $\endgroup$ – Ashoka Lella Aug 24 '17 at 19:26
  • 2
    $\begingroup$ XGBoost Classifier does has a predict_proba option xgboost.readthedocs.io/en/latest/python/python_api.html $\endgroup$ – Paul Bendevis Dec 24 '19 at 19:07

Know I'm a bit late, but to get probabilities from xgboost you should specify multi:softmax objective like this:

xgboost(param, data = x_mat, label = y_mat,nround = 3000, objective='multi:softprob')

From the ?xgb.train:

multi:softprob same as softmax, but output a vector of ndata * nclass, which can be further reshaped to ndata, nclass matrix. The result contains predicted probabilities of each data point belonging to each class.

  • 2
    $\begingroup$ Thanks. How is this loss function different from binary:logistic for binary classification? $\endgroup$ – GeorgeOfTheRF Nov 12 '15 at 7:30
  • 3
    $\begingroup$ It's just a generalization of logistic function for multi-class case, should be no significant difference. $\endgroup$ – cyberj0g Nov 12 '15 at 7:39
  • $\begingroup$ I used it and it worked well, but I had an issue. I'm facing a warning "Parameters: { scale_pos_weight } might not be used.", which comes form the fact I use multi:softprob (source). Why my unbalanced data, it seems that this param is really important (if I go for a classic binary classification), and currently, when I change it, it has no impact because it's not taken into account. $\endgroup$ – BeamsAdept Aug 13 '20 at 8:06

after the prediction

pred_s <- predict(bst, x_mat_s2)

you can get the probability by


If this is a binary classification then pred_s$data includes prob.0, prob.1, response.

So you can get prob.1 by


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.