I have created a csv of data with the following columns: (1) app_key (2) churn, (3)tenure


I have performed the following code in order to drop app_key and churn

test_data = pd.read_csv('/Users/lawrence/Downloads/CHS_August_v4.csv')
test_data_ready = test_data.drop(labels = ["Churn", "app_key"],axis = 1)

I dropped the app_key column so that I could perform an XGBoost operation on the data set. Given that the app_key field is just a customer key, I needed to remove that in order to run by XGBoost operation. You can see the XGBoost operaton below.

  test_model = xgb_model.predict_proba(test_data_ready)[:, 1]
explainer = shap.TreeExplainer(xgb_model)
shap_values = explainer.shap_values(test_data_ready)
shap_output = (pd.DataFrame(data=shap_values, columns=test_data_ready.columns))

Shap_output is a dataframe with several columns of data that represent coefficients. I would like to produce a final output that contains app_key with the corresponding coefficients that have been output from shap_output so I can easily lookup which customers have low coefficients.

I attempted to do this by adding the dataframe of my original csv before dropping the values with my new output, but I'm unsure if this is the correct approach. My main concern is that the rows may not align since I'm unsure if there's any unique sorting. I'm unaware of how to check if sorting may be happening as well. below is the code I used to join my shap_output with my app_key

shap_output['app_key'] = test_data['app_key']

Can someone please let me know if this approach is the best way to ensure that I end up with one output that has the app_key and the corresponding outputs from shap_values? Or is there a better way to connect my app_key label with the corresponding coefficient?

Thank you so much!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.