I'm curious about the assumptions of random forest feature importance.

In this paper, the author says that

"We show that random forest variable importance measures are a sensible means for variable selection in many applications, but are not reliable in situations where potential predictor variables vary in their scale of measurement or their number of categories."

I don't understand what it means by "predictor variables vary in their scale of measurement." Is this referring to the need to standardize data before fitting a random forest model?

  1. Does RF based feature importance require standardization of features?

  2. Does it require fitting a good RF model (ex: pruning), as if I'm making an actual prediction with RF?

  3. What do I do if I have categorical variables and continuous variables at the same time? Is permutation method the only option?

  • $\begingroup$ I am not 100% sure about this, but tree based models don't require scaling as such as that's just another value to split for them instead of the unnormalized one $\endgroup$
    – Aditya
    Nov 14 '19 at 4:26
  • 1
    $\begingroup$ The reasons for this and a solution (implemented at least in R package ranger) are well explained in this paper: ncbi.nlm.nih.gov/pmc/articles/PMC6198850 $\endgroup$ Nov 14 '19 at 8:12
  • $\begingroup$ Too many levels can make random forests act (slightly) pathologically. There are fixes, and healthy systems account for it. I like Boruta. $\endgroup$ Aug 18 '20 at 2:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.