13
$\begingroup$

The below predict function is giving -ve values as well so it cannot be probabilities.

param <- list(max.depth = 5, eta = 0.01,  objective="binary:logistic",subsample=0.9)
bst <- xgboost(param, data = x_mat, label = y_mat,nround = 3000)

pred_s <- predict(bst, x_mat_s2)

I google & tried pred_s <- predict(bst, x_mat_s2,type="response") but it didn't work.

Question

How to predict probabilities instead?

$\endgroup$
  • $\begingroup$ Doesn't it output probabilities by default with the settings you used? I mean: have you examined pred_s and you are certain those are not probabilities? $\endgroup$ – kpb Sep 8 '15 at 11:58
  • $\begingroup$ No its having negative values. Probability should vary between 0 to 1. $\endgroup$ – GeorgeOfTheRF Sep 8 '15 at 12:05
  • $\begingroup$ I don't see any obvious issues. (Although, I'm more familiar with the python wrapper). Have you tried adding outputmargin=F to the predict function? If somehow the outputmargin is set to T, it will return the value before the logistic transformation. $\endgroup$ – inversion Sep 10 '15 at 17:24
  • 1
    $\begingroup$ For Python, you can copy predict_proba implementation from sklearn API: github.com/dmlc/xgboost/blob/master/python-package/xgboost/… $\endgroup$ – Anton Tarasenko Jan 19 '18 at 15:39
12
$\begingroup$

Know I'm a bit late, but to get probabilities from xgboost you should specify multi:softmax objective like this:

xgboost(param, data = x_mat, label = y_mat,nround = 3000, objective='multi:softprob')

From the ?xgb.train:

multi:softprob same as softmax, but output a vector of ndata * nclass, which can be further reshaped to ndata, nclass matrix. The result contains predicted probabilities of each data point belonging to each class.

$\endgroup$
  • 2
    $\begingroup$ Thanks. How is this loss function different from binary:logistic for binary classification? $\endgroup$ – GeorgeOfTheRF Nov 12 '15 at 7:30
  • 3
    $\begingroup$ It's just a generalization of logistic function for multi-class case, should be no significant difference. $\endgroup$ – cyberj0g Nov 12 '15 at 7:39
12
$\begingroup$

Just use predict_proba instead of predict. You can leave the objective as binary:logistic.

$\endgroup$
  • 2
    $\begingroup$ If this were Python and not R, then this answer might be sensible. Wrong language. $\endgroup$ – B_Miner Aug 15 '16 at 18:30
  • 1
    $\begingroup$ oops! thanks @B_Miner. I'm not deleting this answer as it might be helpful for others that will make the same mistake and think we're talking about python.. $\endgroup$ – ihadanny Aug 16 '16 at 8:49
  • $\begingroup$ For me this does not do the trick datascience.stackexchange.com/questions/14527/… $\endgroup$ – Georg Heiler Nov 8 '16 at 7:59
  • 3
    $\begingroup$ xgboost does not have a predict_proba function $\endgroup$ – Ashoka Lella Aug 24 '17 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.