"Binary Encoding" in "Decision Tree" / "Random Forest" Algorithms

Question

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset:

• My dataset has ~170,000 rows
• One of the categoric variables has 1,700 unique values.
• Another one has 3,000 unique values.
• Note that it is not practically possible to group the values of those variables into more aggregate levels.

As a domain expert,I am sure those categorical columns with high cardinalities are strong candidates as predictors. On the other hand, binary encoding surely decreases model's interpretability. Else than interpretability, after binary encoding, is it just alright to build a decision tree / random forest model on the newly formed dataset with new variables which only indicating bits?

Click for a good post on encoding categorical features

Answer 1

In general, it "okay" to apply to binary encode high cardinality datasets. In the sense of it will create numerical features that can be learned by a machine learning model.

However there are often better options, such a label encoding, frequency encoding, target encoding, or embeddings.

It is an empirical question which encoding scheme is best for your specific data and model. The best empirical coding scheme can be found through cross validation.