I'm trying to make a model for a multi-output regression task where $y=(y_1, y_2,..., y_n)$ is a vector rather than a single scalar. I am using Scikit-learn's MultiOutputRegressor method to train and make a model for each $y_i \in y$ separately. My code looks like this:

base_learner = lightgbm.LGBMRegressor(random_state=seed)
estimator = MultiOutputRegressor(regressor)

grid = {
    # hyperpramters to check
    # ...
    # 'random_state': [500],
    'n_estimators': [100, 500],
    'num_leaves': [15, 31, 63],
    'max_depth': [8, 10],
    # 'min_data_in_leaf': [15, 25],
    'feature_fraction': [0.3, 0.4],
    'bagging_fraction': [0.4, 0.5],
    # 'bagging_freq': [100, 200, 400],
    "n_jobs": [-1],
    "verbose": [-1]

gs = GridSearchCV(base_learner, param_grid=grid, scoring=my_custom_score, cv=10)
gs.fit(X_train, y_train)

As you can see, the base-learner for each $y_i$ is of type lightgbm.LGBMRegressor. (By base-learner, I mean each individual leaner used to learn and predict each $y_i$.) I want to do a grid search to pick the best hyperparameters for each base-learner. But I don't know how to pass the list of hyperparameters in the grid variable to the base learners that are wrapped in MultiOutputRegressor. When I run the shown code above, I get the following error:

enter image description here

Do you have any suggestion about how to pass hyperparameters to individual base-learners when one uses MultiOutputRegressor API? (Based on what I see in the error, MultiOutputRegressor itself only takes two parameters which are mainly for a using a leaner not passing hyperparameters to the underlying learners.)


1 Answer 1


I assume you meant GridSearchCV(estimator ..., otherwise there's no wrapping here. You'll need to supply a prefix:

'estimator__n_estimators': [100, 500],
'estimator__bagging_fraction': [0.4, 0.5],

and so on.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.