6
$\begingroup$

Pruning & Truncation

As per my understanding

Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to set a minimum number of training inputs to use on each leaf.

Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances.

  • Can somebody explain the in-detailed implementation of these techniques in GBDT frameworks like XGBoost/LightGBM.

  • Which parameters are used in implementing these techniques?

$\endgroup$

1 Answer 1

3
$\begingroup$

Your understanding is correct. xgboost has nice explanation in the docs.

Reading the original papers is always great idea. Here's one for LGBM and here's one for XGBoost.

As for dessert, here's catboost paper.

Boosting is by far one of the most important concepts in hard machine learning. It's good to know it by heart.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.