0
$\begingroup$

I am trying to understand the model dump output from XGBoost. I would like to step through and see exactly how the model arrived at it's prediction. To simplify I trained a model with 1 tree and 1 max depth, and as expected all records get one of two predictions as it has a single split - the values are {0.5386398434638977, 0.5011891722679138}. However, when I look at the model dump I see the following

booster[0]:
0:[f40<70.5] yes=1,no=2,missing=1
    1:leaf=0.00475667231
    2:leaf=0.154868156

I have no idea how to interpret this in a way that makes sense with the prediction. What am I missing? Thanks!

$\endgroup$
1
$\begingroup$

The scores at leaves measure log-odds, not probabilities. (With more trees, these scores get summed to give a final log-odds approximation, then goes through a sigmoid to get probability approximations.)

And indeed, $1/(1+e^{-0.00475667})\approx 0.501189$.

$\endgroup$
  • $\begingroup$ Makes sense. Thanks! $\endgroup$ – L Xandor Sep 15 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.