I would be reluctant to do too much analysis on the table alone as variable importances can be misleading, but there is something you can do. The idea is to learn the statistical properties of the feature importances through simulation, and then determine how "significant" the observed importances are for each feature. That is, could a large importance for a feature have arisen purely by chance, or is that feature legitimately predictive?
To do this you take the target of your algorithm $y$ and shuffle its values, so that there is no way to do genuine prediction and all of your features are effectively noise. Then fit your chosen model $m$ times, observe the importances of your features for every iteration, and record the "null distribution" for each. This is the distribution of the feature's importance when that feature has no predictive power.
Having obtained these distributions you can compare the importances that you actually observed without shuffling $y$ and start to make meaningful statements about which features are genuinely predictive and which are not. That is, did the importance for a given feature fall into a large quantile (say the 99th percentile) of its null distribution? In that case you can conclude that it contains genuine information about $y$. If on the other hand the importance was somewhere in the middle of the distribution, then you can start to assume that the feature is not useful and perhaps start to do feature selection on these grounds.
Here is a simulation you can do in Python to try this idea out. First we generate data under a linear regression model where only 3 of the 50 features are predictive, and then fit a random forest model to the data. Now that we have our feature importances we fit 100 more models on permutations of $y$ and record the results. Then all we have to do is compare the actual importances we saw to their null distributions using the helper function dist_func
, which calculates what proportion of the null importances are less than the observed. These numbers are essentially $p$-values in the classical statistical sense (only inverted so higher means better) and are much easier to interpret than the importance metrics reported by RandomForestRegressor
. Or, you can simply plot the null distributions and see where the actual importance values fall. In this case it becomes very obvious that only the first three features matter where it may not have been by looking at the raw importances themselves.
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
# number of samples
n = 100
# number of features
p = 50
# monte carlo sample size
m = 100
# simulate data under a linear regression model
# the first three coefficients are one and the rest zero
beta = np.ones(p)
beta[3:] = 0
X = pd.DataFrame(np.random.normal(size=(n, p)),
columns=["x" + str(i+1) for i in range(p)])
y = np.dot(X, beta) + np.random.randn(n)
# fit a random forest regression to the data
reg = RandomForestRegressor()
reg.fit(X, y)
# get the importances
var_imp = (pd.DataFrame({"feature": X.columns,
"beta": beta,
"importance": reg.feature_importances_}).
sort_values(by="importance", ascending=False).
reset_index(drop=True))
# fit many regressions on shuffled versions of y
sim_imp = pd.DataFrame({c: np.empty(m) for c in X.columns})
for i in range(m):
reg.fit(X, np.random.permutation(y))
sim_imp.iloc[i] = reg.feature_importances_
# null distribution function
def dist_func(var, x):
return np.mean(sim_imp[var] < x)