What is Pruning & Truncation in Decision Trees?

Question

Pruning & Truncation

As per my understanding

Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to set a minimum number of training inputs to use on each leaf.

Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances.

• Can somebody explain the in-detailed implementation of these techniques in GBDT frameworks like XGBoost/LightGBM.

• Which parameters are used in implementing these techniques?

Answer 1

Your understanding is correct. xgboost has nice explanation in the docs.

Reading the original papers is always great idea. Here's one for LGBM and here's one for XGBoost.

As for dessert, here's catboost paper.

Boosting is by far one of the most important concepts in hard machine learning. It's good to know it by heart.