4
$\begingroup$

In classification, when we want to get the importance of each variable in the random forest algorithm we usually use Mean Decrease in Gini or Mean Decrease in Accuracy metrics. Now is there a metric which computes the positive or negative effects of each variable not on the predictive accuracy of the model but rather on the dependent variable itself? Something like the beta coefficients in the standard linear regression model but in the context of classification with random forests.

$\endgroup$
1
  • 1
    $\begingroup$ Gini and RF can help one determine Variable importance. For more information on Variable Importance SEE; jstatsoft.org/article/view/v036i11 $\endgroup$
    – mccurcio
    Commented May 3, 2020 at 19:10

1 Answer 1

4
$\begingroup$

With decision trees you cannot directly get the positive or negative effects of each variable as you would with say a linear regression through the coefficients. Its just not the way decision trees work. As you point out, the training process involves finding optimal features and splits at each node by looking at the gini index or the mutual information with the target variable. But no parameters are learnt during the process which we could use for such analysis.

A common tool that is used for this purpose is SHAP. In fact, there is a specific explainer for decision trees based models which is the shap.explainers.Tree. With SHAP you can get both the contribution of the features in pushing towards one or another value of the label, and also an overall view of the contribution of all features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.