If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
No: the trees' results are added together to produce the final score, so combining two models would produce outputs roughly twice as large as desired. (Gradient boosted trees change the target labels being fitted by each tree, so the 101st tree has "reset" the targets when training.)
answered Mar 26, 2020 at 12:47
-
$\begingroup$ The option of adding an already trained model will just ensure that the model will train from those hyper parameters? $\endgroup$ Mar 31, 2020 at 6:19
-
1$\begingroup$ @NextDoorEngineer I'm not sure what you mean. If you mean passing the
xgb_modelparameter totrainorfitmethods, that's training continuation: the new trees are fitted as though you were continuing to train that old model with more trees (not starting from scratch). $\endgroup$– Ben Reiniger ♦Mar 31, 2020 at 16:23
$\begingroup$
$\endgroup$
I don't think Tree models are working in that way. Nodes in XGBoost models depend on features and data you provide to model for training.
However, it is quite possible and better way to combine outputs of XGBoost with another tree model if you want to boost your modals
