Enocding of months for machine learning project

Question

I've been doing a project where I want to use random forest algorithm. There is a column with months, but it is categorical. Was wondering what kind of encoding I should use. I've read that for Random Forest it's not good idea to use OneHotEncoding, when there are more than 10 distinct values. Should I go with LabelEncoding, taking into account that january (1) < february (2) and so on? What would be the impact of that? What would be the impact of using OneHotEncoder in such a case?

Also I have column job with 12 unique values. Same issue - what kind of encoding should I use ?

Answer 1

Trees can performs splits efficiently with ordinal features and handle OHE somewhat worse, so label encoding for months looks like a good start.

As for more complex cases:

1. There's a meaningful order:

• Are the intervals of similar magnitude?
  • Yes: just use label encoding.
  • No: OK, time to worry.

2. No real order:

• Is this a tree/KNN model or the cardinality is large?
  • No: one-hot is a way to go.
  • Yes:
    • Is some information loss acceptable?
      • Yes: feature hashing/bitstrings.
      • No: Leave one out variety of mean target encoding is the default choice. There are other options, like weight of evidence, but you should be cautious about the target leakage.

Answer 2

There is a column with months, but it is categorical. Was wondering what kind of encoding I should use.

If you really want to feed the month as a feature into your model, then the best approach is probably not to encode it, but rather to transform it into a numeric column by calculating:

current_month - your_month

i.e. you're calculating how far back into the past the month is and using that as a feature for your model.

However, I think a better approach would be to not use the month as a feature, but instead transform the monthly data into tabular form. Say you have some feature myColumn with numeic values for each month. You can use myColumn to engineer new features e.g.

the average value of myColumn over the last 3 months, the maximum value of myColumn over the last 6 months, etc.

If myColumn were a categorical feature you could engineer features like

the number of disctinct values in the last 6 months, has this sample contained a partiuclar category over the last 3 months, etc.

The reason I recommend this is that it sounds like your trying to do some time series forecasting using the random forest algorithm. You can definitely do this, but you should transform the time series data into a tabular form if this is the case.

Also I have column job with 12 unique values. Same issue - what kind of encoding should I use ?

If you're transforming the data into tabular form I would use OrdinalEncoder provided by sklearn.

If not then I would try running the model using OneHotEncoder and OrdinalEncoder and simply pick the one with better performance. You could also try dropping columns with high cardinality or you could bin the values in these columns in some manner. I wouldn't use LabelEncoder on the feature set, according to sklearn's documentation it's only for target variables.