XGBOOST with target column has categorical data and features also has categorical data

Question

I have a huge dataset with the categorical columns in features and also my target variable is categorical.

All the values are not ordinal so I think it is best to use one hot encoding.

But I have one issue that my target variable have 90 classes so if I do one hot encoding there will be 90 columns as the target columns and it will become to much complex.

But as all the values are not ordinal can I apply one hot encoding for the features categorical columns and label encoder for the target variable?

Thanks

Answer 1

What you have is a classic case of high dimensional categorical data. Basically you have a lot of categorical features in your input, along with the target variable having a lot of classes.

In this case I would suggest not using OneHotEncoder as it will further increase your dimensionality and result in poor predictions.

Also you mentioned that there are no ordinal features. Hence using OrdinalEncoder won't make sense too.

There are a lot of different types of encoders, not just OneHot and Ordinal and depending on your type of features, you can choose which one suits the best. category-encoders has a list of various encoders you can check out.

I would suggest CatBoostEncoder or LeaveOneOutEncoder as they usually perform best on high dimensional data but try out all and see which works.

Cheers!

Answer 2

I don't know your dataset and your target. From my experience I can say that it is possible. In some case I applied label encoding to a categorical classes and hot encoding to the features of the dataset.

You could provide an example of your dataset and explain your target if you want a specific answer.