1
$\begingroup$

I've been doing a project where I want to use random forest algorithm. There is a column with months, but it is categorical. Was wondering what kind of encoding I should use. I've read that for Random Forest it's not good idea to use OneHotEncoding, when there are more than 10 distinct values. Should I go with LabelEncoding, taking into account that january (1) < february (2) and so on? What would be the impact of that? What would be the impact of using OneHotEncoder in such a case?

Also I have column job with 12 unique values. Same issue - what kind of encoding should I use ?

$\endgroup$
1
  • 1
    $\begingroup$ I don't think there is any issue with OneHotEncoding features with more than 10 distinct values. The important thing is to keep a rows to columns ratio large enough. If that ratio is still large (say > 100) after OneHotEncoding then you are good to go $\endgroup$
    – Anatole
    Sep 20, 2022 at 9:45

2 Answers 2

1
$\begingroup$

Trees can performs splits efficiently with ordinal features and handle OHE somewhat worse, so label encoding for months looks like a good start.

As for more complex cases:

  1. There's a meaningful order:
  • Are the intervals of similar magnitude?
  1. No real order:
  • Is this a tree/KNN model or the cardinality is large?
    • No: one-hot is a way to go.
    • Yes:
      • Is some information loss acceptable?
        • Yes: feature hashing/bitstrings.
        • No: Leave one out variety of mean target encoding is the default choice. There are other options, like weight of evidence, but you should be cautious about the target leakage.
$\endgroup$
2
  • $\begingroup$ So what if I am going to use Random forest and the cardinality is 12 for 2 columns ? $\endgroup$ Sep 20, 2022 at 10:32
  • $\begingroup$ For months, start with label encoding. For jobs, try mean target encoding. $\endgroup$
    – dx2-66
    Sep 20, 2022 at 10:39
1
$\begingroup$

There is a column with months, but it is categorical. Was wondering what kind of encoding I should use.

If you really want to feed the month as a feature into your model, then the best approach is probably not to encode it, but rather to transform it into a numeric column by calculating:

current_month - your_month

i.e. you're calculating how far back into the past the month is and using that as a feature for your model.

However, I think a better approach would be to not use the month as a feature, but instead transform the monthly data into tabular form. Say you have some feature myColumn with numeic values for each month. You can use myColumn to engineer new features e.g.

the average value of myColumn over the last 3 months, the maximum value of myColumn over the last 6 months, etc.

If myColumn were a categorical feature you could engineer features like

the number of disctinct values in the last 6 months, has this sample contained a partiuclar category over the last 3 months, etc.

The reason I recommend this is that it sounds like your trying to do some time series forecasting using the random forest algorithm. You can definitely do this, but you should transform the time series data into a tabular form if this is the case.

Also I have column job with 12 unique values. Same issue - what kind of encoding should I use ?

If you're transforming the data into tabular form I would use OrdinalEncoder provided by sklearn.

If not then I would try running the model using OneHotEncoder and OrdinalEncoder and simply pick the one with better performance. You could also try dropping columns with high cardinality or you could bin the values in these columns in some manner. I wouldn't use LabelEncoder on the feature set, according to sklearn's documentation it's only for target variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.