8
$\begingroup$

How do we use one hot encoding if the number of values which a categorical variable can take is large ?

In my case it is 56 values. So as per usual method I would have to add 56 columns (56 binary features) in the training dataset which will immensely increase the complexity and hence the training time.

So how do we deal with such cases ?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Look into feature hashing $\endgroup$
    – Emre
    Nov 3, 2015 at 0:38
  • $\begingroup$ What algorithm do you use? SGD can process hundreds of thousands of features on hundreds of thousands data rows in a few minutes on a laptop. $\endgroup$
    – Diego
    Feb 4, 2016 at 22:08

4 Answers 4

Reset to default
10
$\begingroup$

If you really care about the number of dimensions, you still can try to apply a dimensionality reduction algorithm, such as PCA (Principal Component Analysis) or LDA (Linear Discriminant Analysis), after your one hot encoding.

But know that "56 features" isn't really large and it's highly common in the industry to have thousands, millions or even billions of features.

Improve this answer
$\endgroup$
1
  • $\begingroup$ is it normal to have 50 features and 60 categories using RNNs ? $\endgroup$
    – Boppity Bop
    Apr 24, 2019 at 7:33
2
$\begingroup$

You could try reducing the dimmension of the 56 dummy resulting features, if you have some categories that represent a small proportion compared to the majority by labeling them the same.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Is there any way to find out what features should be clubbed together, because in my case all the features are more or less equally important.. ? $\endgroup$
    – mach
    Oct 3, 2015 at 19:34
  • $\begingroup$ Try the ones with small frequency! $\endgroup$
    – Alexandru Daia
    Oct 4, 2015 at 5:46
  • $\begingroup$ ...or you could make the values more granular. e.g. Northeast, mid-atlantic, etc. instead of the 50 states. Is there any sort of domain knowledge you could use to merge certain factors together? $\endgroup$
    – user13684
    Nov 2, 2015 at 21:10
  • 1
    $\begingroup$ How do you know that all of the categorical values are equally important?What was your methodology... did you do Pearson correlation with the target, lasso regression, decision tree, ...? How are you assessing feature importance? $\endgroup$
    – AN6U5
    Nov 3, 2015 at 17:45
1
$\begingroup$

It depends on the problem you are working on. If number of categorical variables is very large, it is better to use label encoding. But the label encoding should be meaningful i.e. the categories which are close to each other should get similar labels. Let's say you are creating a model where you have a feature Month. But there is a periodicity in your target variable, i.e. every x months, say 3 months, the trends are similar. Now it does not make sense to use labels 1, 2, ... 12 for months, instead, it is better to use 0, 1, 2, 0, 1, 2.... such labels. So Jan is 0, Feb is 1, Mar is 2 and again Apr is 0 and so on.

You can use LabelEncoder of sklearn.preprocessing for this problem. But it does not take care of the semantics as I mentioned. For that, you can do some manual label encoding.

Improve this answer
$\endgroup$
-1
$\begingroup$

When there are large number of categorical variables, it is advisable to do one versus rest.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.