As mentioned in the question, it is easy to interpret the meaning of features in algorithms like simple decision trees. But in the case of ensemble methods that are known to average/modify features, are these results still sensible interpretable/usable to argument about the feature(s)?


2 Answers 2


During recent years, the most successful feature attribution method from has been the SHAP values.

SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allocation with local explanations using the classic Shapley values from game theory and their related extensions

One of the most famous implementations is Tree Explainer

Tree SHAP is a fast and exact method to estimate SHAP values for tree models and ensembles of trees

The paper originally appeared in Nature ( here in Arxiv): Explainable AI for Trees: From Local Explanations to Global Understanding.

In case you want to extend and see more related literature you can have a look at the appendix "Methods 5 Previous Global Explanation Methods for Tree". They provide the previous state of the art of feature-relevance methods.

The exact algorithm that answers your question is in the Method 10, it explains how TreeShap is computed.


Feature importances and the interpretation of the meaning of features are entirely different things.

The importance of a feature shows how strongly the feature influences the answer. If the importance is low, the answer only slightly depends on the feature. If the importance is high, the answer is highly influenced by the feature.

The interpretation of the meaning of a feature relates to the question of how ( = in which form, in which way) the feature influences the answer.

I will illustrate it here with a simple example. Because, you ask about ensemble methods, I will use two ensemble methods: random forest and gradient boosting:

from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor

Let's take a 1000 samples of 3 features $x_0$, $x_1$, $x_2$, each with values between 0 and 1.

x0 = np.random.uniform(low=0.0, high=1.0, size=(1000,1))
x1 = np.random.uniform(low=0.0, high=1.0, size=(1000,1))
x2 = np.random.uniform(low=0.0, high=1.0, size=(1000,1))

X = np.concatenate((x0, x1, x2), axis=1)

Let the target variable $y$ depend on $x_0$ and $x_1$ but not on $x_2$, e.g. $y = -1 + 2x_0+sin(6\pi x_1)$

y = -1 + 2 * x0 + np.sin(6 * np.pi * x1)

These features look as follows enter image description here

Calculating feature importances with random forest,

reg = RandomForestRegressor()
reg.fit(X, y.ravel())


[0.43457062 0.55951066 0.00591872]

and with gradient boosting, reg = GradientBoostingRegressor():

[4.02569166e-01 5.96839179e-01 5.91654881e-04]

Two features $x_0$, and $x_1$ have similar importances, as can be seen from the picture above, while the importance of $x_2$ is much less (ideally, it should be zero).

Let's scale one feature relative to another, and see what happens to feature importances.

y = -0.2 + 0.4 * x0 + np.sin(6 * np.pi * x1)

enter image description here

Feature importances according to random forest:

[0.01887166 0.97918728 0.00194106]

Feature importances according to gradient boosting:

[2.08344663e-02 9.79105292e-01 6.02418447e-05]

The second feature is more important. Let's scale them differently.

y = -1 + 2 * x0 + 0.2 * np.sin(6 * np.pi * x1)

enter image description here

Feature importances according to random forest:

[0.9545089  0.03933793 0.00615317]

Feature importances according to gradient boosting:

[9.48205203e-01 5.17843116e-02 1.04852064e-05]

Now the first feature is more important.

We could easily evaluate feature importances with ensemble algorithms, and they make sense. The feature that greater influences the answer is more important.

From the pictures, we also see the meaning of each feature. However, we cannot get the meaning easily from the ensemble algorithms. Usually, for many algorithms, it is easier to evaluate feature importances than interpret the meaning of features.

If the question of feature meanings can be answered by analyzing a certain algorithm, that algorithm is interpretable. Otherwise, it will be called uninterpretable, black box, or something like that. The feature is still there, its role doesn't change, no mater if a human can interpret it or not.

Even if you take the same class of algorithms, say, decision trees, you can build different decision trees for the same problem, depending on the method of tree growing. I illustrated it on the XOR example in this answer: link where two trees for the same problem are constructed, one being interpretable, the other is not.

As far as the modification of features is concerned, every algorithm modifies features. Even the decision tree does that. When it chooses a subset of a feature, it is equivalent to the multiplication of the feature by an indicator function $I(x)$: if $x<x_0$ then $I(x) = 0$ else $I(x) = 1$. Other algorithms modify features differently. Ensemble methods are nor more harmful to features than any other.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.