I have a data with continous variables and categorical variables. I am using Random Forest and have made my continues variables Gaussian by transformation and have standardized it. Should categorical variables be done the same?


Afaik, once you deal with the categorical variables you end up having several columns where the values are either true/false (or 0/1). So, I do not see how making them Gaussian would help.

  • $\begingroup$ Yeah you are right. And what if I have an indicator variable which can have 0 or 1, then do I need to standardize it? $\endgroup$ – user1825567 Aug 27 '17 at 9:50
  • 1
    $\begingroup$ By standarizing do you mean scaling? My understanding is that random forest does not require feature scaling. $\endgroup$ – Juan Antonio Gomez Moriano Aug 27 '17 at 23:25

You can just convert them into factors if your random forest algorithm is accepting that .Usually one hotting the categorical variables are not suggested Here is the an article gives information about categorical variables in Random Forest

  • $\begingroup$ Very informative +1 $\endgroup$ – user1825567 Sep 8 '17 at 12:11

There's another approach to dealing with categorical variables that is called target/impact encoding.

In this scheme the idea is to encode the feature using a single float column in which the value is the average of the target variable over all rows that share the category. This is especially useful for tree based models since it imposes an order relationship within the feature (ie values to the right of the category have higher mean response than values to the left) and it makes it easier divide the predictor space.

Here's a nice explanation of the subject:

And here's a link to the paper that originally proposed the encoding: http://helios.mm.di.uoa.gr/~rouvas/ssi/sigkdd/sigkdd.vol3.1/barreca.pdf

There's some more details to avoid estimating the mean in categories with low counts and also there's another model, CatBoost, proposing a solution to the biasing introduced by this encoding, but in my experience it's a simple and very useful way to encode high cardinality categorical variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.