How to set parameters to search in scikit-learn GridSearchCV

Question

I want to use scikit-learn's GridSearchCV to optimise a BaggingClassifier that uses a support vector classifier (SVC). I want the grid search to search over parameters for both the BaggingClassifier and the SVC.

I have tried this setup:

svc_pipe = Pipeline([
     ('svc', SVC(probability=True)),
])
pipe = Pipeline([
     ('bag', BaggingClassifier(svc_pipe, no_estimators=50)),
])

params = {
    'bag__bootstrap_features' : [True, False],
    'bag__svc__kernel': ['linear', 'rbf'],
    'bag__svc__decision_function_shape': ['ovo', 'ovr']
}

rnd_search = GridSearchCV(pipe, param_grid=params)

but I get this error:

ValueError: Invalid parameter svc for estimator BaggingClassifier(base_estimator=Pipeline(memory=None,
     steps=[('svc', SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=True, shrinking=True,
  tol=0.001, verbose=False))]),
         bootstrap=True, bootstrap_features=True, max_features=1.0,
         max_samples=1.0, n_estimators=50, n_jobs=-1, oob_score=False,
         verbose=0, warm_start=False). Check the list of available parameters with `estimator.get_params().keys()`.

Can someone show me what I have done wrong?

Answer 1

There is a typo in pipe, no_estimators should be n_estimators.

To address your problem, if you run the following piece of code:

for param in rnd_search.get_params().keys():
    print(param)

This will show you how the parameters are passed to different parts of the pipeline, the parameters of interest are:

• bag__base_estimator__svc__kernel
• bag__base_estimator__svc__decision_function_shape

So you were almost there, you were just missing base_estimator__ in the svc pipeline parameters. All you need to do is change the svc parameters like so:

params = {
    'bag__bootstrap_features' : [True, False],
    'bag__base_estimator__svc__kernel': ['linear', 'rbf'],
    'bag__base_estimator__svc__decision_function_shape': ['ovo', 'ovr']
}