1
$\begingroup$

I have a XGBoost model with the following parameters

xgbc_final = XGBClassifier(objective="multi:softprob",
                           num_class = 2,max_depth = 60,
                           n_estimators = 512,
                           reg_lambda = 0.1214,
                           alpha = 0.9131,
                           gamma = 0,
                           colsample_bytree = 0.7,
                           colsample_bylevel = 0.8,
                           colsample_bynode = 0.7,
                           subsample = 0.6,
                           learning_rate = .01,
                           min_child_weight = 14,
                           random_state = 2020,
                           eval_metric = 'auc',
                           verbosity = 1)

Here, I only have n_estimator = 512 but I noticed that when I try to print a decision tree greater than 511, I still get a plot

plot_tree(xgbc_final, num_trees=900)

I expected an error for n_estimator greater than 511(if the trees are index from 0)

Can anyone explain why it's spitting out trees for number greater than 512?

$\endgroup$

1 Answer 1

0
$\begingroup$

It's because you're doing multiclass classification, and xgboost does that by building parallel models for each class. So the total number of trees is actually $512\cdot (\text{# classes})$.

I don't know what order those trees come in. plot_tree calls to_graphviz which calls (after maybe model.get_booster) model.get_dump, which calls some c-level stuff I won't try to track down.

$\endgroup$
1
  • $\begingroup$ Thanks for the clarification! If I were to guess, for class 0, index=511 is the last tree and for class 1, index=1023 is the last tree. $\endgroup$
    – Itminan
    Mar 24, 2023 at 22:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.