I have a data set something like this:

data = [['Alex',10,13,1,0],['Bob',11,14,12,0],['Clarke',13,15,13,1],['bob',12,15,1,1]]
df = pd.DataFrame(data,columns = ["dealer","x","y","z","loss"])

I am trying to predict binary column loss, I have done this xgboost model. I got Overall feature importance. Now I need top 5 most important features dealer wise.

How to do that?

I have tried to use lime package but it is only working for Random forest.

If I get Feature importance for each observation(row) then also I can compute the feature importance dealer wise.

kindly help

Desired Output : enter image description here

  • $\begingroup$ Are you looking for which of the dealer categories is most predictive of a loss=1 over the entire dataset? In the example above dealer is text which makes it categorical and you handled that somehow which is not explained above. $\endgroup$ – Craig Jan 7 at 11:23
  • $\begingroup$ @Craig I have edited the question. I am looking for Dealer-wise most important variables which is helping me predict loss. $\endgroup$ – Nirali Khoda Jan 7 at 11:44
  • $\begingroup$ You should create 3 datasets sliced on Dealer. Then get the FI for each feature. This seems the only meaningful approach. $\endgroup$ – 10xAI Jan 7 at 15:39
  • $\begingroup$ cross-posted at stats.stackexchange.com/q/503861/232706 $\endgroup$ – Ben Reiniger Jan 7 at 16:08
  • $\begingroup$ @10xAI You mean to say i need to build multiple models ? as I have really less data I am not able to do that. $\endgroup$ – Nirali Khoda Jan 8 at 5:42

If you use a per-observation explanation, you could just average (or aggregate in some other way) the importances of features across the samples for each Dealer.

For example, using shap to generate the per-observation explanation:

import pandas as pd
from sklearn.linear_model import LogisticRegression
import shap

data = [['Alex',10,13,1,0],['Bob',11,14,12,0],['Clarke',13,15,13,1],['Bob',12,15,1,1]]
df = pd.DataFrame(data, columns=["dealer","x","y","z","loss"])

lr = LogisticRegression()
lr.fit(df[['x', 'y', 'z']], df['loss'])

# Whatever explainer you prefer:
explainer = shap.explainers.Permutation(lr.predict_proba, df[['x', 'y', 'z']])
shap_values = explainer(df[['x', 'y', 'z']])

# get just the explanations for the positive class
shap_values = shap_values[...,1]

shap_df = pd.DataFrame(abs(shap_values.values))
shap_df.columns = ['x_shap', 'y_shap', 'z_shap']
shap_df['dealer'] = df['dealer']


dealer x_shap y_shap z_shap
Alex 0.260427 0.140054 0.075176
Bob 0.106593 0.069035 0.091146
Clarke 0.268328 0.083706 0.085807
  • $\begingroup$ Hey, do you have any example of shap per observation explanation as I saw that first but i couldn't find any example on that. $\endgroup$ – Nirali Khoda Jan 8 at 5:44
  • $\begingroup$ Thanks This helped a lot :) $\endgroup$ – Nirali Khoda Jan 11 at 12:22

What you are looking for is - "When Dealer is X, how important is each Feature."

You can try Permutation Importance.

  • Can be used on fitted model
  • It is Model agnostic
  • Can be done for Test data too.

Slice X, Y in parts based on Dealer and get the Importance separately.

Shown for California Housing Data on Ocean_Proximity feature

from sklearn.inspection import permutation_importance

for val in x_train.ocean_proximity.unique():        # Loop on column value
    x = x_train.loc[x_train.ocean_proximity==val,:] # Slices X
    y = y_train.loc[x_train.ocean_proximity==val]   # Slices Y
    result = permutation_importance(model, x, y, n_repeats=5, random_state=0)
    result = pd.DataFrame(result.importances_mean, index= cols)
    # print, sorted top 4 Features

Note - The importance value for each feature with this test and "Impurity decreased" approach are not comparable.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.