1
$\begingroup$

The one hot encoder adds more columns to the data, one for each category in the encoded feature. In the example below, the column City was transformed into 4 other columns. Suppose a Decision Tree is ran on a dataset the below is part of and City_Chicago and City_New_York appear to be in top most important features while City_Detroit and City_SanFrancisco in the least important. Would there be any problem if I drop City_Detroit and City_SanFrancisco from my dataset, but keep City_Chicago and City_New_York or do I need to keep all city features as they are part of one initial feature?

|---------------------|------------------|-------------|---------------|---------------|
|      City           | City_SanFrancisco| City_Detroit| City_New_York | City_Chicago  |
|---------------------|------------------|-------------|---------------|---------------|
|     San Francisco   |         1        |      0      |       0       |     0         |
|---------------------|------------------|-------------|---------------|---------------|
|     Detroit         |         0        |      1      |       0       |     0         |
|---------------------|------------------|-------------|---------------|---------------|
|     New York        |         0        |      0      |       1       |     0         |
|---------------------|------------------|-------------|---------------|---------------|
|     Chicago         |         0        |      0      |       0       |     1         |
|---------------------|------------------|-------------|---------------|---------------|
Improve this question
$\endgroup$
1
  • $\begingroup$ Some people seem to prefer to keep all our none: see "group lasso." $\endgroup$
    – Ben Reiniger ♦
    Commented Jun 17, 2020 at 16:45

1 Answer 1

Reset to default
1
$\begingroup$

I think you can keep as many as you want, and it'll be alright. Sometimes it's even worth to delete the very rare classes to have more stable features.

In addition, for linear regression, you shouldn't include all of them, as you might have a collinearity issue.

To sum up, no problem with not keeping them all.

Improve this answer
$\endgroup$
1
  • $\begingroup$ good point on rare classes. Ok, I get that it will be ok if I delete some of them. $\endgroup$
    – callmeGuy
    Commented Jun 17, 2020 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.