1
$\begingroup$

I have asked this question here, but seems no one was interested in it:

https://stats.stackexchange.com/questions/550994/if-a-feature-has-already-split-will-it-hardly-be-selected-to-split-again-in-the

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree? It is motivated by the fact that for the heavy correlated features in a single tree, usually only one of them will be selected to split as their uncertainty will remain few after a splitting. Now in Gradient Boosting Tree, is residual similar with the uncertainty?

Currently, I happened to how heavily correlated features affects the feature importance selected by Gradient Boosting Tree. I guess the result is that Gradient Boosting Tree will only select the importance from one of correlated features just like LASSO.

$\endgroup$

1 Answer 1

1
$\begingroup$

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree?

It's harder yes, but common. In the same tree it can not happen in the same point. In a subsequent tree it can.

It is motivated by the fact that for the heavy correlated features in a single tree, usually only one of them will be selected to split as their uncertainty will remain few after a splitting. Now in Gradient Boosting Tree, is residual similar with the uncertainty?

If you are trying to understand gradient boosting I would not mix it with uncertainty. There are several ways to compute the uncertainty with gradient boosting, NGBoost is one but not the only.

Your question needs further retunnign to properly be answered. You are mixing highly correlated features, with uncertainty. You need to clarify them before attempting to answer this question.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.