0
$\begingroup$

I'm using pipeline to transform data and predict model and I want to apply SHAP after that. However, when I apply it, it returns SHAP chart just fine, but the name of the feature are like feature 1, feature 2, etc - like the image bellow.

How can I get the real feature name in shap?

My code:

def pipeline(categoricas_all,numericas_all, model):
    encoder = OneHotEncoder() #apenas para categoricas com baixa cardinalidade

    imputer_num = SimpleImputer(strategy = 'median')
    imputer_cat = SimpleImputer(strategy = 'most_frequent')
    
    numeric_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy = 'median'))
    # ,('scaler', StandardScaler())
    ])

    categorical_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy = 'most_frequent'))
    ,('encod', encoder)
    ])

    preprocessor = ColumnTransformer(
     transformers=[
    ('num', numeric_transformer, numericas_all)
    ,('cat', categorical_transformer, categoricas_all)
    ])
    
    pipe = Pipeline(steps=[('preprocessor', preprocessor),('classifier', model)])
    return preprocessor, pipe


processor, pipe = pipeline(categoricas_all,numericas_all, item)
pipe.fit(X_train, y_train)

explainer = shap.Explainer(pipe["classifier"])
data_transformation = pipe['preprocessor'].transform(X_test)

shap_values = explainer(data_transformation)

shap.plots.waterfall(shap_values[0])

enter image description here

$\endgroup$

1 Answer 1

0
$\begingroup$

probably a bit late, but still.

In sklearn, Pipeline/ColumnTransformer (and other) have usually function get_feature_names_out() returning feature names after transformation (so matching the shape of transformed data) and shap.Explainer takes feature_names as argument, so in your case:

explainer = shap.Explainer(pipe["classifier"], feature_names=processor.get_feature_names_out())
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.