When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg.

params = {
  'num_leaves': trial.suggest_int('num_leaves', 2, 128),
  'objective': 'regression', 
  'learning_rate': 0.01,
  'min_data_in_leaf': trial.suggest_int('min_data_in_leaf', 2, 20),
  'boosting': 'gbdt', 
  'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10),
  'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10), 
  'bagging_freq': 5,
  'bagging_fraction': trial.suggest_loguniform('bagging_fraction', 0.1, 1), 
  'feature_fraction': trial.suggest_uniform('feature_fraction', 0.4, 1),
  'metric': 'l2',
  'verbose_eval': -1,
  'seed': 0,
  'max_depth': trial.suggest_int('max_depth', 1, 6),
  'extra_trees': True,

Because I already state that I want to use the gbdt boosting method, why is it possible to also define parameters for bagging (bagging_freq, bagging_fraction)? I thought that bagging and boosting were two different ensemble methods. The docs briefly discuss the role of bagging, but I'm still confused about the idea of using both bagging and boosting at the same time.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.