1
$\begingroup$

There a lot of information on how to handle categorical variables when preprocessing data for ML classification. However, I cannot find any feedback on how to handle categorical variables, where each sample can belong to more than one label.

I'm working on bug detection classifier. I've got many features like who contributed to the source code. There are about 200 unique labels and creating so many dummy variables makes my model overfit.

So are there any alternatives for this method. Something like target-based encoders (ex. CatBoost)

$\endgroup$
1
  • $\begingroup$ Have you tried One-hot-encoders? $\endgroup$
    – Full Array
    Jan 3 at 2:55

2 Answers 2

0
$\begingroup$

Just having 200 unique labels and using MultiLabelBinarizer does not automatically mean overfitting. Overfitting is an empirical question. The number features and observations relative to the complexity of the algorithm effect the chance of overfitting.

$\endgroup$
0
$\begingroup$

You need to venture into the art of "feature engineering". You noted you already tried "dummies" and maybe one-hot-encoders. You noted you have too many features that may or may not be good predictors. You also noted you have trained your model, and it appears as if though it is overfitting. As a result, your next best option is "feature engineering".

For feature engineering to be successful, you have to have a strategy and a well defined goal. It is not as simple as putting your data into a pre-backed function implementation of sklearn or similar AI/ML libraries and letting it work automatically in autopilot. With feature engineering, you drive, and you may need the assistance of a subject matter expert (SME), to ensure your data science/AI goal produces a valuable model (to send it to production if applicable).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.