I have a categorical feature that I one-hot encoded and used in my XGBoost model, but it consistently underperforms as a predictor compared to the other predictors.
Then I created a new variable that contains the same kind of information that the categorical feature has...
Imagine I'm interested in predicting price of a house and the categorical feature is the town that it is in, and the numerical feature is a feature I generated ranking the town's relative expensiveness, based on some prior knowledge of the towns.
TownA - 100 TownB - 40 TownC - 65 TownD - 15
Now all of a sudden the new numerical variable that was directly derived from the categorical one is outperforming. Is this because XGBoost just works better with numerical variables and maybe some predictive capability is lost when I one-hot encode the variable?