0
$\begingroup$

I'm trying to do multi-class classification on a labeled dataset with purely categorical features. There are around 30 features in total. 3 of the features in particular have around 100 unique values (high cardinality). What would the general approach or things to keep in mind while tackling such a dataset?

I'm currently halfway into frequency encoding (of high cardinality features) followed by RFE or Information gain for feature selection. Could use some unsupervised outlier detection technique?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I think depending on the model you want to use for the data, the categorical features may not matter. For tree models, you can just encode the feature values as is, and the model will sort itself out. For linear algebra models what you could do is train the model with a lookup table for the parameters, such that if you do one-hot encoding you can reduce a bunch of redundant operations. I $\endgroup$
    – timmy1691
    Commented Mar 12 at 19:19
  • $\begingroup$ For NN you can include a layer of embedding learning, which will learn the right way to embed your one-hot encoded features as part of training. $\endgroup$
    – Cryo
    Commented Mar 13 at 4:21

1 Answer 1

Reset to default
1
$\begingroup$

Quick answer: try CatBoost, and any other tree-based models. Tree-based models can handle categories as-is without any special treatment/encoding, and usually give state-of-the-art result.

If feature encoding is needed and cardinality is high, try feature hashing. It can also handle previously-unseen categories in future.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.