XGBoost prints trees beyond n_estimator

Question

I have a XGBoost model with the following parameters

xgbc_final = XGBClassifier(objective="multi:softprob",
                           num_class = 2,max_depth = 60,
                           n_estimators = 512,
                           reg_lambda = 0.1214,
                           alpha = 0.9131,
                           gamma = 0,
                           colsample_bytree = 0.7,
                           colsample_bylevel = 0.8,
                           colsample_bynode = 0.7,
                           subsample = 0.6,
                           learning_rate = .01,
                           min_child_weight = 14,
                           random_state = 2020,
                           eval_metric = 'auc',
                           verbosity = 1)

Here, I only have n_estimator = 512 but I noticed that when I try to print a decision tree greater than 511, I still get a plot

plot_tree(xgbc_final, num_trees=900)

I expected an error for n_estimator greater than 511(if the trees are index from 0)

Can anyone explain why it's spitting out trees for number greater than 512?

Answer 1

It's because you're doing multiclass classification, and xgboost does that by building parallel models for each class. So the total number of trees is actually $512\cdot (\text{# classes})$.

I don't know what order those trees come in. plot_tree calls to_graphviz which calls (after maybe model.get_booster) model.get_dump, which calls some c-level stuff I won't try to track down.