4
$\begingroup$

I want to evaluate the importance of each of the features of a 2000x60 dataset in a classification problem with random forest.

The most widely used ones apparrently are:

  • Cross Entropy-Information Gain
  • Gini Importance (SkLearn implementation with feature_importances_)
  • Mean Squared Error (H2O implementation with h2o.varimp)

I have also found a rather concise overview of some other metrics for variables' importance at random forests at this research paper.

These are the following:

  • Altmann
  • Boruta
  • Permutation
  • Recurrent relative variable importance
  • Recursive feature elimination
  • Vita
  • VSURF

Has anyone used these and which one was the most informative for his/her model?

Do you have any other metrics of this kind for variable importance at random forests?

$\endgroup$
2
  • $\begingroup$ Have you seen github.com/slundberg/shap? A quick intro.: medium.com/civis-analytics/…. $\endgroup$ Aug 31 '18 at 5:57
  • $\begingroup$ Thank you for your comment. As we can see there are multiple method to something like this. I am more interested to know which ones have been used personally by you or by data scientists in general and which ones performed the best. Have you used any of the aforementioned methods? How did they perform? $\endgroup$
    – Outcast
    Aug 31 '18 at 10:55
0
$\begingroup$

One of the most commonly used methods is permutation feature importance which is decrease in a model score when a single feature value is randomly shuffled. The result is ranked ordered list of features and with each one having a distributional estimates. It works very well with Random Forests.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.