The below predict function is giving -ve values as well so it cannot be probabilities.

param <- list(max.depth = 5, eta = 0.01,  objective="binary:logistic",subsample=0.9)
bst <- xgboost(param, data = x_mat, label = y_mat,nround = 3000)

pred_s <- predict(bst, x_mat_s2)

I google & tried pred_s <- predict(bst, x_mat_s2,type="response") but it didn't work.


How to predict probabilities instead?

  • $\begingroup$ Doesn't it output probabilities by default with the settings you used? I mean: have you examined pred_s and you are certain those are not probabilities? $\endgroup$
    – kpb
    Commented Sep 8, 2015 at 11:58
  • $\begingroup$ No its having negative values. Probability should vary between 0 to 1. $\endgroup$ Commented Sep 8, 2015 at 12:05
  • $\begingroup$ I don't see any obvious issues. (Although, I'm more familiar with the python wrapper). Have you tried adding outputmargin=F to the predict function? If somehow the outputmargin is set to T, it will return the value before the logistic transformation. $\endgroup$
    – inversion
    Commented Sep 10, 2015 at 17:24
  • 1
    $\begingroup$ For Python, you can copy predict_proba implementation from sklearn API: github.com/dmlc/xgboost/blob/master/python-package/xgboost/… $\endgroup$ Commented Jan 19, 2018 at 15:39

4 Answers 4


Just use predict_proba instead of predict. You can leave the objective as binary:logistic.

  • 4
    $\begingroup$ If this were Python and not R, then this answer might be sensible. Wrong language. $\endgroup$
    – B_Miner
    Commented Aug 15, 2016 at 18:30
  • 2
    $\begingroup$ oops! thanks @B_Miner. I'm not deleting this answer as it might be helpful for others that will make the same mistake and think we're talking about python.. $\endgroup$
    – ihadanny
    Commented Aug 16, 2016 at 8:49
  • $\begingroup$ For me this does not do the trick datascience.stackexchange.com/questions/14527/… $\endgroup$ Commented Nov 8, 2016 at 7:59
  • 6
    $\begingroup$ xgboost does not have a predict_proba function $\endgroup$ Commented Aug 24, 2017 at 19:26
  • 2
    $\begingroup$ XGBoost Classifier does has a predict_proba option xgboost.readthedocs.io/en/latest/python/python_api.html $\endgroup$ Commented Dec 24, 2019 at 19:07

Know I'm a bit late, but to get probabilities from xgboost you should specify multi:softmax objective like this:

xgboost(param, data = x_mat, label = y_mat,nround = 3000, objective='multi:softprob')

From the ?xgb.train:

multi:softprob same as softmax, but output a vector of ndata * nclass, which can be further reshaped to ndata, nclass matrix. The result contains predicted probabilities of each data point belonging to each class.

  • 2
    $\begingroup$ Thanks. How is this loss function different from binary:logistic for binary classification? $\endgroup$ Commented Nov 12, 2015 at 7:30
  • 3
    $\begingroup$ It's just a generalization of logistic function for multi-class case, should be no significant difference. $\endgroup$
    – cyberj0g
    Commented Nov 12, 2015 at 7:39
  • $\begingroup$ I used it and it worked well, but I had an issue. I'm facing a warning "Parameters: { scale_pos_weight } might not be used.", which comes form the fact I use multi:softprob (source). Why my unbalanced data, it seems that this param is really important (if I go for a classic binary classification), and currently, when I change it, it has no impact because it's not taken into account. $\endgroup$
    – Adept
    Commented Aug 13, 2020 at 8:06

After the prediction:

pred_s <- predict(bst, x_mat_s2)

You can get the probability by:


If this is a binary classification then pred_s$data includes prob.0, prob.1, response.

So you can get prob.1 by:


Many years late, but noticed this; and this is how I do it:

pred_s <- predict(bst, x_mat_s2, type="prob")

Typically then I then work with probability of my upper class "1":


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.