Encoding of high cardinality multi-label categorical feature?

Question

This is the problem of binary classification: "1" - the subscriber is a driver (belongs to the segment of drivers), "0" - the subscriber is not a driver (does not belong to the segment of drivers).

The files tabular_data.csv and hashed_feature.csv ̶ here are descriptive characteristics for 4084 subscribers ("ID" is the subscriber ID). The train.csv file is data about the target label (subscriber id and corresponding binary target).

The tabular_data.csv file contains numerical data on the subscriber's activity for 12 periods. • period - period number (consecutive periods, 1 - the newest); • id - subscriber ID; • feature_0 - feature_49 - data on the subscriber's activity in the corresponding period.

The hashed_feature.csv file is a set of hashed values of one categorical variable for the subscriber. • id - subscriber ID; • feature_50 - hash from the value of the categorical variable.

I faced the following problems:

1. How should I encode feature_50 for ML algorithm? This feature contains approximately 5k unique values and each user can have almost 1k of these values. As I think one hot encoding is not very helpful here, but what technique should I use in this case?
2. NA filling strategy for feature_0 - feature_49 (I've only tried pandas filling strategies)
3. Make use of period variable? I would appreciate any help.

Answer 1

When you have high cardinality I suggest you two options (one doesn't exclude the other):

• Aggregate least frequent categories into one called other
• Use Neural Network with an embedding layer for high cardinality categories. In case you don't know what an embedding is, it is just a table that map each category into a vector of k features (k your choice)