0
$\begingroup$

I am working with a pure categorical data set. And some classes have more than 100 unique values. I could not find any appropriate encoding possibility. So I created a SQL table, where each value got its ID. Then I extracted the IDs and used it in ML Classification. However the results are poor. So anyone has an idea how to encode such values better?

$\endgroup$
0
$\begingroup$

Even though you have 100 unique values - Do you see a pattern where perhaps 80 of those unique values appear only once or twice - and the remainder 20 occur much more often.

In that case - you may want to encode the 80 with a special "Other" value. By doing this you lose information - by not being able to distinguish between the 80 categories - but perhaps it doesn't matter to your problem.

The other option is - You can convert your categorical data into an integer (perhaps the ID approach you are talking about ?) and run a Decision Tree model (e.g. Random Forests) on it. My experience with Random Forests is that they work pretty well on such data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.