2
$\begingroup$

I'm curious about the assumptions of random forest feature importance.

In this paper, the author says that

"We show that random forest variable importance measures are a sensible means for variable selection in many applications, but are not reliable in situations where potential predictor variables vary in their scale of measurement or their number of categories."

I don't understand what it means by "predictor variables vary in their scale of measurement." Is this referring to the need to standardize data before fitting a random forest model?

  1. Does RF based feature importance require standardization of features?

  2. Does it require fitting a good RF model (ex: pruning), as if I'm making an actual prediction with RF?

  3. What do I do if I have categorical variables and continuous variables at the same time? Is permutation method the only option?

$\endgroup$
  • $\begingroup$ I am not 100% sure about this, but tree based models don't require scaling as such as that's just another value to split for them instead of the unnormalized one $\endgroup$ – Aditya Nov 14 at 4:26
  • $\begingroup$ The reasons for this and a solution (implemented at least in R package ranger) are well explained in this paper: ncbi.nlm.nih.gov/pmc/articles/PMC6198850 $\endgroup$ – crazysantaclaus Nov 14 at 8:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.