Is there a way to output feature importance based on the outputted class?

Question

I'm running a random forest classifier in Python (two classes). I am using the feature_importances_ method of the RandomForestClassifier to get feature importances.

It provides a nice visualization of importances but it does not offer insight into which features were most important for each class. For example, it may be for class 1 that some feature values were important, whereas for class 2 some other feature values were more important.

Is it possible to split feature important based on the predicted class?

Answer 1

You can use sklearn.inspection.permutation_importance for this purpose

Split your test data by predicted labels and pass to the method

Code example

from sklearn.inspection import permutation_importance
import matplotlib.pyplot as plt
import seaborn as sns

model = RandomForestClassifier()
model.fit(x_train, y_train)

def plot_feature_importances(model, x, y, title):
    result = permutation_importance(model, x, y, n_repeats=100, random_state=0)
    df = pd.DataFrame({'feature_name': x.columns, 'feature_importance': result.importances_mean})
    plt.figure(figsize=(8, 2))
    sns.barplot(data=df, x='feature_importance', y='feature_name')
    plt.title(title)
    plt.show()

plot_feature_importances(model, x_test, y_test, 'All test data')

y_pred = model.predict(x_test)
plot_feature_importances(model, x_test[y_pred == 1], y_test[y_pred == 1], 'Predicted as "1"')
plot_feature_importances(model, x_test[y_pred == 0], y_test[y_pred == 0], 'Predicted as "0"')

Answer 2

• A matchbox • Concentrated foods • 50 meters of nylon rope • A silk parachute • A heater running on solar energy • 2 45 caliber pistols • A case of powdered milk • 2 tanks of 50 kg of oxygen each • A celestial map of the lunar constellations • A self-inflating lifeboat • A magnetic compass • 25 liters of water • A medical kit and hypodermic syringes • Light signals • A transceiver running on solar energy (MF frequency)