I am training an XGBoost regression model on a feature set $X$ that includes a feature $x_k$ with high cardinality (~100). First, I am using one-hot-encoding to convert $x_k$ and then split the set into
testing sets. The model seems to work well. So far everything's pretty standard.
The problem arises when I make predictions on unseen data. In the unseen data, the cardinality of $x_k$ is slightly different. To put into perspective, say the unique values of $x_k$ in training & testing set was $\lbrace aa,ab,ac,...,ay,az \rbrace$. In the unseen data, the cardinality set is $\lbrace aa, ab, ...., ay \rbrace $. So $az$ does not appear in unseen data. To be able to make predictions with the model I had, I have to have the same columns in the unseen data as in
training. To remedy this, I tried two ways:
- Create a new column in unseen data that will correspond to the dummy variable $az$, and assign 0 to all rows. Here, my logic was: "OK, apparently $az$ is observed rarely so creating a column with all zero values would make the sets consistent and should not hurt accuracy so bad". It did.
- Train the XGBoost model by deleting the column corresponding to the dummy variable $az$. You can think of it as applying one-hot-encoding with a cardinality set $\lbrace aa, ab, ...., ay \rbrace $. In this case, the accuracy in
testingset and on unseen data were good. However, the problem is I needed to train the model again, and I do not want to do so each time I want to make predictions.
I know there are methods other than one-hot-encoding for categorical features, but I am curious on how to approach to this specific issue.