1
$\begingroup$

I am trying to understand the model dump output from XGBoost. I would like to step through and see exactly how the model arrived at it's prediction. To simplify I trained a model with 1 tree and 1 max depth, and as expected all records get one of two predictions as it has a single split - the values are {0.5386398434638977, 0.5011891722679138}. However, when I look at the model dump I see the following

booster[0]:
0:[f40<70.5] yes=1,no=2,missing=1
    1:leaf=0.00475667231
    2:leaf=0.154868156

I have no idea how to interpret this in a way that makes sense with the prediction. What am I missing? Thanks!

$\endgroup$

1 Answer 1

2
$\begingroup$

The scores at leaves measure log-odds, not probabilities. (With more trees, these scores get summed to give a final log-odds approximation, then goes through a sigmoid to get probability approximations.)

And indeed, $1/(1+e^{-0.00475667})\approx 0.501189$.

$\endgroup$
1
  • $\begingroup$ Makes sense. Thanks! $\endgroup$
    – L Xandor
    Sep 15, 2019 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.