Encoding features with big amount of classes

Question

Is it worth to encode features with big amount of classes ( such as 60 )? Or should I leave it as it is ?

Answer 1

big amount of classes are called High-cardinality refers to columns with values that are very uncommon or unique.

Dealing on High Cardinality depends upon the data/use case/model,

The Following are the methods we can use to handle High Cardinality:

1. Drop (According to business case)
2. Embed with frequency/count.
3. Target encoding/ CatBoost encodings.
4. Reducing Cardinality by using a simple Aggregating function

Refer below links

• Encoding of categorical variables with high cardinality
• How to encode a categorical feature with high cardinality?

Answer 2

Usually when we have Categorical variables, we do one hot encoding to convert to numerical data and use in model. If we have n classes we get n variables.

Now, High Cardinality Variables are those categorical variables which have large number of classes and doing one hot encoding may lead to a high dimensional data which we try to avoid.

But in a lot of these cases these high cardinality variables may have a lot of information and we want to use them as part of training data. To do so we can apply following techniques :

      1. Try to drop the category which in less frequent or has a frequency of less then 1% or you can encode it with some special category say("rare"). You can use some business hypothesis to do that
      
      2. Replace the Categorical Variables by using Smoothed Weight of evidence encoding

      3. Use algorithms like CatBoost which encodes using Target encoding.

Like any other variables, if cardinal variables seems some relation with Target variables it may seem sense to encode it otherwise not.

For a more detailed analysis please refer to this link