I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model). Someone suggested to use target-encoding (mean/median of the target of each item) together with xgboost.

While I understand how this new feature would improve a linear model (or GMM'S in general) I do not understand how this approach would fit into a tree-based model (Regression Trees, Random Forest, Boosting).

Given the feature is used for splitting, items with a mean below and above the splitting value are separated. But since the final prediction is based on the mean of each leaf I do not see why this feature is helpful.

Therefore my question is whether it is sensible to use target encoding in this setting? If this is the case I would be thankful for a short explanation?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.