6
$\begingroup$

Just what the title says. Suppose I know the feature that I want to be used for splitting for the root node in a tree model in XGBoost; is there a way for me to tell XGBoost to split on this feature at the root node?

$\endgroup$
2
  • $\begingroup$ I don't know the answer to your question, but I'm curious why you want to do this. $\endgroup$
    – tom
    Oct 22, 2019 at 20:42
  • 1
    $\begingroup$ @tom I have a categorical feature that can take one of several values, and I'm considering whether I want to make a single model and let XGBoost deal with this feature on its own, or if I want to make a different model for each possible value of this feature. The latter option is tedious and time consuming, so if I can tell XGBoost just to always split on that feature first then that should serve as an easy way of testing the "many different models" case. $\endgroup$
    – bgav
    Oct 22, 2019 at 21:18

3 Answers 3

1
$\begingroup$

I do not have the whole answer, but I believe that xgboost allows warm-starting, which implies starting from a provided point.

xgb.train(…,xgb_model=model_to_start_from)
$\endgroup$
1
  • $\begingroup$ Warm starting in xgb, AFAIK, only works for adding more base learners, not extending existing ones. $\endgroup$
    – Ben Reiniger
    Dec 18, 2019 at 14:09
1
$\begingroup$

It's hard to prove a negative, but I think the answer is no. Especially in xgboost compared to other tree-model packages, it's rather hard to even access the base trees, let alone modify their build.

$\endgroup$
0
$\begingroup$

Could you enforce this using feature interactions? Maybe create two sets of features with only that one feature in common? I’m not sure if it will guarantee that that shared feature is the one on which the first split is made.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.