0
$\begingroup$

I'm using the feature_importances_ attribute in the random forest classifier of scikit-learn to plot the importances of each feature. However, I'd like to plot these importances non-normalized. I have searched around how to do this, but there doesn't seem to be an easy method how to do this. I tried manually:

temp = [t.tree_.compute_feature_importances(normalize=True) for t in clf.estimators_]
arr = np.array(temp)
arr2=[]
for i in range(19):
    arr2.append(sum(arr[:,i]))
arr3 = np.array(arr2)
indices = np.argsort(arr3)[::-1]
indices.reshape(1,-1)

plt.figure()
plt.title("Feature importances")
plt.bar(range(X_train.shape[1]), arr3[indices],
        color="r", align="center")
plt.xticks(range(X_train.shape[1]), indices)
plt.xlim([-1, X_train.shape[1]])
plt.show()

Which gives these results for normalize=True and normalize=False respectively. (the normalize parameter seems to be inverted somehow...?) normalize=True normalize=False

For reference, this is the result from using feature_importances_ (the error-bars are not relevant to the question): enter image description here

$\endgroup$
0
$\begingroup$

It seems you are normalizing the Gini dip for every tree and then summing those values.
You should do the normalization at the end after the sum.

Result might vary depending upon the overall numbers in the two scenarios.
See this dummy example

import numpy as np
# Three Tree and 3 Features. F1 sum across Trees = 50, F2 = 48, F3 = 20
gini_dip = np.array([[20, 15, 5], [20, 15, 5], [10, 18, 10]]) 

# Normaliation before sum
gini_dip_n = np.array([((elem-min(elem))/(max(elem)-min(elem))) for elem in gini_dip])
fi_1 = gini_dip_n.sum(axis=0)

# Normaliation After sum
gini_dip_sum = gini_dip.sum(axis=0)
fi_2 = (gini_dip_sum - gini_dip_sum.min())/(gini_dip_sum.max() - gini_dip_sum.min())

fi_1, fi_2

(array([2. , 2.33333333, 0. ]),
array([1. , 0.93333333, 0. ]))

F2 is best in scenario #1 and F1 is best in scenario #2

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.