Grid-search for a multi-output regression task using Scikit-learn's API

Question

I'm trying to make a model for a multi-output regression task where $y=(y_1, y_2,..., y_n)$ is a vector rather than a single scalar. I am using Scikit-learn's MultiOutputRegressor method to train and make a model for each $y_i \in y$ separately. My code looks like this:

base_learner = lightgbm.LGBMRegressor(random_state=seed)
estimator = MultiOutputRegressor(regressor)

grid = {
    # hyperpramters to check
    # ...
    # 'random_state': [500],
    'n_estimators': [100, 500],
    'num_leaves': [15, 31, 63],
    'max_depth': [8, 10],
    # 'min_data_in_leaf': [15, 25],
    'feature_fraction': [0.3, 0.4],
    'bagging_fraction': [0.4, 0.5],
    # 'bagging_freq': [100, 200, 400],
    "n_jobs": [-1],
    "verbose": [-1]
}

gs = GridSearchCV(base_learner, param_grid=grid, scoring=my_custom_score, cv=10)
gs.fit(X_train, y_train)

As you can see, the base-learner for each $y_i$ is of type lightgbm.LGBMRegressor. (By base-learner, I mean each individual leaner used to learn and predict each $y_i$.) I want to do a grid search to pick the best hyperparameters for each base-learner. But I don't know how to pass the list of hyperparameters in the grid variable to the base learners that are wrapped in MultiOutputRegressor. When I run the shown code above, I get the following error:

Do you have any suggestion about how to pass hyperparameters to individual base-learners when one uses MultiOutputRegressor API? (Based on what I see in the error, MultiOutputRegressor itself only takes two parameters which are mainly for a using a leaner not passing hyperparameters to the underlying learners.)

Answer 1

I assume you meant GridSearchCV(estimator ..., otherwise there's no wrapping here. You'll need to supply a prefix:

'estimator__n_estimators': [100, 500],
'estimator__bagging_fraction': [0.4, 0.5],

and so on.