2
$\begingroup$

I have been searching for a while and I just can't find any indication. When people talk about iterations in algorithms like XGBoost or LightGBM, or Catboost, do they mean how many decision trees i.e. base learners will be built? I.e. XGboost m=100 means the algorithm will build a total of 100 base learners, each calculating and optimizing towards the residual value of the previous prediction?
Or is it more like 1 epoch in deep learning?

$\endgroup$
4
$\begingroup$

Your first interpretation is correct. One base learner will be added per boosting iteration/round and that is probably what people are referring to when talking about iterations.

From wiki:

One natural regularization parameter is the number of gradient boosting iterations M (i.e. the number of trees in the model when the base learner is a decision tree).

Iterations take place in other parts of the algorithm, for instance in the gradient descent, but I don't think that is what is discussed if it is simply referred to as "iterations".

This was really helpful when I was trying to understand GBMs

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.