0
$\begingroup$

I am trying to run the below code to extract the feature importances of my random forest, but I'm getting the following error TypeError: init() got an unexpected keyword argument 'randomforestclassifier__max_depth'. Can anyone tell me what is wrong?

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score
from sklearn.pipeline import make_pipeline
from imblearn.over_sampling import RandomOverSampler
from imblearn.under_sampling import RandomUnderSampler
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import RepeatedStratifiedKFold
from sklearn.metrics import f1_score

x, y = make_classification(n_samples=10000, weights=[0.99], flip_y=0)
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.2, random_state = 1)

paramgrid_rf = {'n_estimators': [500],
                'max_depth': [4],
                'random_state': [0],
                'max_features': ['sqrt']
                }

imba_pipeline_rf = make_pipeline(RandomOverSampler(sampling_strategy=0.35, random_state=0),
                             RandomUnderSampler(sampling_strategy=0.9, random_state=0),
                             RandomForestClassifier())

new_params = {'randomforestclassifier__' + key: paramgrid_rf[key] for key in paramgrid_rf}
cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=0)
grid_imba_rf = GridSearchCV(imba_pipeline_rf, param_grid=new_params, cv=cv, scoring='f1',
                        return_train_score=True)
scores = cross_val_score(imba_pipeline_rf, x_train, y_train, scoring='f1', cv=cv, n_jobs=-1)
grid_imba_rf.fit(x_train, y_train)
y_pred_rf = grid_imba_rf.predict(x_test)
print('F1 score on validation data: ', f1_score(y_test, y_pred_rf))

rf_final = RandomForestClassifier(**grid_imba_rf.best_params_).fit(x_train_res,y_train_res)
rf_final.feature_importances_
$\endgroup$
2
$\begingroup$

Your grid search dictionary contains the argument names with the pipeline step name in front of it, i.e. 'randomforestclassifier__max_depth'. Instead, the RandomForestClassifier has argument names without the pipeline step name, i.e. max_depth. You therefore need to remove the first part of the string which denotes the name of the step in your original pipeline. You can do this using a dictionary comprehension:

# original
{'randomforestclassifier__max_depth': 4, 'randomforestclassifier__max_features': 'sqrt', 'randomforestclassifier__n_estimators': 500, 'randomforestclassifier__random_state': 0}

# splitting the key on '__' and take only the last part
{k.split("__")[-1]: v for k, v in grid_imba_rf.best_params_.items()}
# {'max_depth': 4, 'max_features': 'sqrt', 'n_estimators': 500, 'random_state': 0}

This changes one line in the original script to:

rf_final = RandomForestClassifier(
    **{k.split("__")[-1]: v for k, v in grid_imba_rf.best_params_.items()}
).fit(x_train,y_train)
rf_final.feature_importances_
$\endgroup$
2
  • $\begingroup$ Thanks that worked like a charm. I thought the "raondomforestclassifier__<param>" approach will work because it did for XGBClassifier and LGBMClassifier. Also I read in the scikit learn documentation that: set_params(**params) - The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object. $\endgroup$ – lostwanderer Jan 29 at 10:37
  • $\begingroup$ nice answer, upvote! $\endgroup$ – German C M Jan 29 at 10:50
0
$\begingroup$

Welcome to the community. By taking a quick look at your code, it seems to be that your RandomForestClassifier instance is receiving randomforestclassifier__max_depth as input param, instead of just the sklearn defined param name max_depth.

The error seems to come from your definition of new_params when adding 'randomforestclassifier__'.

Make sure you pass to the RF classifier the defined params with the correct names.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.