Error while using lightGBM's cv() function for a regression problem

Question

I am trying to use lightGBM's cv() function for tuning my model for a regression problem. My main model is lightgbm.LGBMRegressor(). However, I am encountering the errors which is a bit confusing given that I am in a regression mode and NOT classification mode. Below are the code snippet and part of the trace.

param_grid = {
    'class_weight': [None, 'balanced'],
    'num_leaves': list(range(30, 150)),
    'learning_rate': list(np.logspace(np.log(0.005), np.log(0.2), base = np.exp(1), num = 1000)),
    'subsample_for_bin': list(range(20000, 300000, 20000)),
    'min_child_samples': list(range(20, 500, 5)),
    'reg_alpha': list(np.linspace(0, 1)),
    'reg_lambda': list(np.linspace(0, 1)),
    'colsample_bytree': list(np.linspace(0.6, 1, 10)),
    'objective': 'regression'
 } 
lgb_train = lgb.Dataset(X_train,y_train)
cv_results = lgb.cv(params, lgb_train, num_boost_round = 10000, nfold = 10, metrics = 'rmse', shuffle=False,
           early_stopping_rounds = 100, verbose_eval = False, seed = 50 )

Part of the trace of the error I am getting is

---------------------------------------------------------------------------
LightGBMError                             Traceback (most recent call last)
<ipython-input-84-3f76f5345fda> in <module>
      1 # Perform cross validation with 10 folds
      2 cv_results = lgb.cv(params, lgb_train, num_boost_round = 10000, nfold = 10, metrics = 'rmse', shuffle=False,
----> 3            early_stopping_rounds = 100, verbose_eval = False, seed = 50, )

/anaconda3/envs/py36/lib/python3.6/site-packages/lightgbm/engine.py in cv(params, train_set, num_boost_round, folds, nfold, stratified, shuffle, metrics, fobj, feval, init_model, feature_name, categorical_feature, early_stopping_rounds, fpreproc, verbose_eval, show_stdv, seed, callbacks)
    456     cvfolds = _make_n_folds(train_set, folds=folds, nfold=nfold,
    457                             params=params, seed=seed, fpreproc=fpreproc,
--> 458                             stratified=stratified, shuffle=shuffle)
    459 
    460     # setup callbacks

/anaconda3/envs/py36/lib/python3.6/site-packages/lightgbm/engine.py in _make_n_folds(full_data, folds, nfold, params, seed, fpreproc, stratified, shuffle)
    315         else:
    316             tparam = params
--> 317         cvbooster = Booster(tparam, train_set)
    318         cvbooster.add_valid(valid_set, 'valid')
    319         ret.append(cvbooster)

/anaconda3/envs/py36/lib/python3.6/site-packages/lightgbm/basic.py in __init__(self, params, train_set, model_file, silent)
   1552                 train_set.construct().handle,
   1553                 c_str(params_str),
-> 1554                 ctypes.byref(self.handle)))
   1555             # save reference to data
   1556             self.train_set = train_set

/anaconda3/envs/py36/lib/python3.6/site-packages/lightgbm/basic.py in _safe_call(ret)
     44     """
     45     if ret != 0:
---> 46         raise LightGBMError(decode_string(_LIB.LGBM_GetLastError()))
     47 
     48 

LightGBMError: Unknown objective type name: r

I don't understand the error. Moreover, if I remove the 'objective':'regressor', then we are getting the error: y variable is only 1 class which seems to me to be referring to a classifier.

Any help would be great.

Thanks

Answer 1

Answer

In the param_grid dictionary, pass every hyper-parameter as array e.g., in your code above, you missed-out to place 'objective': 'regression' part of param_grid as an array. Even though, you have are using single value place it as array i.e. 'objective': ['regression']

Therefore, you can update below piece of code in your script and re-run. It will remove the error. `

param_grid = {
    'class_weight': [None, 'balanced'],
    'num_leaves': list(range(30, 150)),
    'learning_rate': list(np.logspace(np.log(0.005), np.log(0.2), base = np.exp(1), num = 1000)),
    'subsample_for_bin': list(range(20000, 300000, 20000)),
    'min_child_samples': list(range(20, 500, 5)),
    'reg_alpha': list(np.linspace(0, 1)),
    'reg_lambda': list(np.linspace(0, 1)),
    'colsample_bytree': list(np.linspace(0.6, 1, 10)),
    'objective': ['regression'] # <<< CHANGE REQUIRED TO remove ERROR
 }