8
$\begingroup$

How do we use one hot encoding if the number of values which a categorical variable can take is large ?

In my case it is 56 values. So as per usual method I would have to add 56 columns (56 binary features) in the training dataset which will immensely increase the complexity and hence the training time.

So how do we deal with such cases ?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Look into feature hashing $\endgroup$ – Emre Nov 3 '15 at 0:38
  • $\begingroup$ What algorithm do you use? SGD can process hundreds of thousands of features on hundreds of thousands data rows in a few minutes on a laptop. $\endgroup$ – Diego Feb 4 '16 at 22:08
10
$\begingroup$

If you really care about the number of dimensions, you still can try to apply a dimensionality reduction algorithm, such as PCA (Principal Component Analysis) or LDA (Linear Discriminant Analysis), after your one hot encoding.

But know that "56 features" isn't really large and it's highly common in the industry to have thousands, millions or even billions of features.

Improve this answer
$\endgroup$
1
  • $\begingroup$ is it normal to have 50 features and 60 categories using RNNs ? $\endgroup$ – Boppity Bop Apr 24 '19 at 7:33
2
$\begingroup$

You could try reducing the dimmension of the 56 dummy resulting features, if you have some categories that represent a small proportion compared to the majority by labeling them the same.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Is there any way to find out what features should be clubbed together, because in my case all the features are more or less equally important.. ? $\endgroup$ – mach Oct 3 '15 at 19:34
  • $\begingroup$ Try the ones with small frequency! $\endgroup$ – Alexandru Daia Oct 4 '15 at 5:46
  • $\begingroup$ ...or you could make the values more granular. e.g. Northeast, mid-atlantic, etc. instead of the 50 states. Is there any sort of domain knowledge you could use to merge certain factors together? $\endgroup$ – user13684 Nov 2 '15 at 21:10
  • 1
    $\begingroup$ How do you know that all of the categorical values are equally important?What was your methodology... did you do Pearson correlation with the target, lasso regression, decision tree, ...? How are you assessing feature importance? $\endgroup$ – AN6U5 Nov 3 '15 at 17:45
-1
$\begingroup$

When there are large number of categorical variables, it is advisable to do one versus rest.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.