0
$\begingroup$

I am trying to perform K-means clustering on a dataset, and one of my categorical features has 96 possible options. Would this be too many features for one variable to have? The alternative would be to either attempt to convert it to a numerical variable through weight of evidence, or simply drop it. What do you guys think?

$\endgroup$

1 Answer 1

1
$\begingroup$

K-means really suffers in high dimensions, so adding 96 new features to the dataset would be tough. One option would be to one-hot-encode the categorical features then use some dimensionality reduction technique before clustering.

One other thought is to bin the categories so instead of 96, you have maybe 4 or 5. This might be a good choice if the majority of the dataset fit into just a few of the categories. e.g. if 90% of the examples have category A, B, C, or D, then you could keep those four categories and combine the rest into a catch-all "Other" category.

convert it to a numerical variable through weight of evidence

Weight-of-evidence is often most useful for binary classification problems. (The weight tells you how much the feature contributes to a "positive" vs. "negative" outcome). If you use weight-of-evidence, just keep in mind that the clustering will be biased towards the variable you compute WoE against.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.