4
$\begingroup$

I am using the xgboost library. My system runs a cronjob each night, where it pulls the data from the database and trains the model. However, I would like to remove the re-training of the model again and again, and just fine-tune it with any new data that came in the database. In sklearn's implemantation (http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html) one could use warm_start option, what about xgboost ?

$\endgroup$
3
  • $\begingroup$ Are you using xgboost bound to Python? The command-line version definitely does what you want: model_in [default=NULL] path to input model, needed for test, eval, dump, if it is specified in training, xgboost will continue training from the input model - but I don't know whether and how the Python bindings support this option. $\endgroup$ Sep 18, 2015 at 12:25
  • $\begingroup$ yes I am using the python version, I will check if something like this is supported. $\endgroup$ Sep 18, 2015 at 23:30
  • $\begingroup$ It's not supported in the python module, but there is a fork that shows the simple change needed to enable this. github.com/Far0n/xgboost/commit/… $\endgroup$
    – inversion
    Sep 19, 2015 at 14:36

3 Answers 3

2
$\begingroup$

I see that in the current version of python wrapper of xgboost you can specify file name or existing xgboost model (class Booster) in train function.

$\endgroup$
2
$\begingroup$

It should be noted that XGBoost makes optimal splits assuming it has access to the entire dataset, so you would likely be losing out on some predictive power by updating rather than retraining (though of course this may be worth the lessened computational cost).

See an interesting discussion on this XGBoost github issue

$\endgroup$
0
$\begingroup$

You can do this by the argument xgb_model. The splits are fixed but the leave values are updated by your new training data.

And you can increase the tree splitting using new data after the original ones.

Here is a minimal example as follows,

gbm_t = xgb.train(params_t, xgb_train_t, num_boost_round = num_round, evals= watchlist_t, xgb_model=gbm)
$\endgroup$
1
  • $\begingroup$ I don't think this is how the training continuation works. Rather, I think additional trees are built, appending to the list of base estimators (after running your (new?) training data through the existing trees to get their pseudo-residuals. $\endgroup$
    – Ben Reiniger
    May 13, 2021 at 14:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.