0
$\begingroup$

Hope you are holding on tight, Christmas soon!

**Updates: I'm only able to get the results in probability but I want it in pure binary (0 or 1). I tried using .predict instead of predict_proba(x_test) but then I get the error code "LIME does not currently support classifier models without probability scores. If this conflicts with your use case, please let us know:". Some of those that I have linked have used the predict and it has worked. Also I'm getting this output error (but the tables are showing at least) "UserWarning: X does not have valid feature names, but RandomForestClassifier was fitted with feature names warnings.warn("

I've tried following these guides https://www.kaggle.com/code/vikumsw/explaining-random-forest-model-with-lime and https://www.kaggle.com/code/srushtipujari/predicting-german-credit-default/notebook . As I prefer not to completely copy their code I would like an alternative if you have one or suggestions on how to improve my code :)! This is the dataset

Below you will find my whole code:

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics
import matplotlib.pyplot as plt 

import numpy as np ; np.random.seed(sum(map(ord, "aesthetics")))
import pandas as pd

from sklearn.datasets import make_classification  
from sklearn.metrics import classification_report,confusion_matrix, roc_curve, roc_auc_score, auc, accuracy_score
from sklearn.model_selection import ShuffleSplit,train_test_split, cross_val_score, GridSearchCV
from sklearn.preprocessing import OneHotEncoder, LabelEncoder, label_binarize, StandardScaler, MinMaxScaler

import seaborn 
seaborn.set_context('notebook') 
seaborn.set_style(style='darkgrid')


 
df = pd.read_csv("file.csv", header=None)

df.shape

df.columns = ['Status of checking account', 'Duration in months', 'Credit history', 'Purpose', 'Credit amount', 
         'Savings account/bond', 'Present employment since', 'Installment rate in percentage of disposable income', 'Personal status and sex', 'Other debtors', 
         'Present residence since', 'Property', 'Age in years', 'Other installment plans', 'Housing', 
         'Number of existing credits', 'Job', 'Number of people providing maintenance for', 'Telephone', 'Foreign worker', 'Credit risk']

df.info()

from sklearn.datasets import make_classification 
df['Credit risk'].replace([1,2], [1,0], inplace=True)

from sklearn.preprocessing import StandardScaler

Ndata = pd.DataFrame(StandardScaler().fit_transform(df[['Duration in months', 'Credit amount', 'Installment rate in percentage of disposable income','Present residence since', 'Age in years', 'Number of existing credits', 'Number of people providing maintenance for']]))

Numdata = df.select_dtypes(include="int")

for col in df.select_dtypes(include="object"):    
    df[col] = df[col].astype("category")

x = pd.get_dummies(df.select_dtypes(include="category"))
x.head(20)


data_clean = pd.concat([Numdata, x], axis = 1)

print(data_clean.shape)

# Unscaled, unnormalized data
x_2 = data_clean.drop('Credit risk', axis=1)
y_2 = data_clean['Credit risk']


from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(x_2, y_2, test_size=0.2, random_state=1)

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, confusion_matrix

RF = RandomForestClassifier(criterion="entropy", random_state=1)
RF.fit(x_train, y_train)
RF_pred = RF.predict(x_test)
print(classification_report(y_test, RF.predict(x_test)))


from imblearn.over_sampling import SMOTE


sm = SMOTE(sampling_strategy='auto')
X_train_bal, y_train_bal = sm.fit_resample(x_train, y_train)



unique, counts = np.unique(y_2, return_counts=True)
print(dict(zip(unique, counts)))
unique, counts = np.unique(y_train_bal, return_counts=True)
print(dict(zip(unique, counts)))


X_train_bal = pd.DataFrame(X_train_bal, columns=X_train_bal.keys())
y_train_bal = pd.DataFrame(y_train_bal)

RF2 = RandomForestClassifier(criterion="entropy", random_state=1)
RF2.fit(X_train_bal, y_train_bal.values.ravel())
RF2_pred = RF2.predict(x_test)
print(classification_report(y_test, RF2.predict(x_test)))

import lime
import lime.lime_tabular

categorical_features = np.argwhere(np.array([len(set(X_train_bal.values[:,x]))
for x in range(X_train_bal.values.shape[1])]) <= 10).flatten()

explainer = lime.lime_tabular.LimeTabularExplainer(X_train_bal.values, 
                                                   feature_names=X_train_bal.columns.values.tolist(), 
                                                   categorical_features=categorical_features, 
                                                   verbose=True, mode='regression')

predict_fn_rf = lambda x: RF2.predict_proba(x).astype(float)
X = X_train_bal.values
explainer = lime.lime_tabular.LimeTabularExplainer(X,feature_names = X_train_bal.columns,class_names=['Default','No Default'])

i = 49
 
exp = explainer.explain_instance(x_test.iloc[i], predict_fn_rf, num_features=5)
exp.show_in_notebook(show_table=True)

ERROR IF I TRY TO MAKE THE RF2.predict_proba(x) into a 0 or 1 binary instead of probability: 

Cell In [31], line 3
      1 i = 49
----> 3 exp = explainer.explain_instance(x_test.iloc[i], predict_fn_rf, num_features=5)
      4 exp.show_in_notebook(show_table=True)

File ~\OneDrive\Documents\Anaconda\lib\site-packages\lime\lime_tabular.py:361, in LimeTabularExplainer.explain_instance(self, data_row, predict_fn, labels, top_labels, num_features, num_samples, distance_metric, model_regressor)
    359 if self.mode == "classification":
    360     if len(yss.shape) == 1:
--> 361         raise NotImplementedError("LIME does not currently support "
    362                                   "classifier models without probability "
    363                                   "scores. If this conflicts with your "
    364                                   "use case, please let us know: "
    365                                   "https://github.com/datascienceinc/lime/issues/16")
    366     elif len(yss.shape) == 2:
    367         if self.class_names is None:

NotImplementedError: LIME does not currently support classifier models without probability scores. If this conflicts with your use case, please let us know: https://github.com/datascienceinc/lime/issues/16

I would really love to get this one to work since I think it's really nice to get that interpretability aspect in there for the random forest model.

Cheers!

$\endgroup$
2
  • $\begingroup$ Welcome to DS Stack Exchange. It would be nice to have the error you have with your code. $\endgroup$ Dec 7, 2022 at 19:01
  • $\begingroup$ Added the error at the bottom that I get when trying to apply the [:,1] to the RF2.predict_proba(x) to convert the results into binary outcomes. @Nicolas Martin $\endgroup$
    – Ostlimpa
    Dec 8, 2022 at 0:03

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.