I'm trying to tune some parameters in XGBoost and read a lot about "...makes to model more conservative". Can somebody explain me what the word conservative means in this case?
I can imagine that it learns slower (more similar observations needed) and is less prone to overfitting? I'm not sure if my assumption is correct though.
Example from the docu:
eta [default=0.3, alias: learning_rate]
Step size shrinkage used in update to prevents overfitting. After each boosting step, we can directly get the weights of new features, and eta shrinks the feature weights to make the boosting process more conservative.
gamma [default=0, alias: min_split_loss]
Minimum loss reduction required to make a further partition on a leaf node of the tree. The larger gamma is, the more conservative the algorithm will be.
lambda [default=1, alias: reg_lambda]
L2 regularization term on weights. Increasing this value will make model more conservative.
and many more..