Questions tagged [gradient-boosting-decision-trees]

Filter by
Sorted by
Tagged with
0
votes
0answers
35 views

What to do as I'm getting 100% accuracy on all models? [closed]

Code Link - https://colab.research.google.com/drive/1tjJsaIu-Y58kuIWy3tQL-jombyBinxpn?usp=sharing I tried GradientboostedClassifier, XGBClassifier, LogisticRegression and i'm getting 100% accuracy on ...
0
votes
0answers
12 views

Uncertainty prediction in Gradient Boosted Tree based Quantile Regression

For an application, I am using a Gradient boosting Tree based quantile regression model (LightGBM, Catboot) to predict the 5th percentile of the target variable. The model predicts point estimates, ...
1
vote
1answer
24 views

Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?

I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
2
votes
1answer
20 views

What if root of a such tree is pruned in xgboost?

Extreme Gradient Boosting stops to grow a tree if $\gamma$ is greater than impurity reduction given as eq (7) (see below) , what does happen if tree's root has a negative impurity? I think there is no ...
1
vote
1answer
94 views

XGBoost - Imputing Vs keeping NaN

What is the benefit of imputing numerical or categorical features when using DT methods such as XGBoost that can handle missing values? This question is mainly for when the values are missing not at ...
1
vote
1answer
28 views

Tree complexity and gamma parameter in xgboost

According to xgboost paper, regularization is given by: $$\Omega(f) = \gamma T + \lambda || w||^2$$ where $\gamma$ is the complexity of a tree (i.e., number of leaves in the tree). The parameter ...
0
votes
1answer
33 views

Chossing between gradient boosting algorithms

I just stepped in machine learning competitions and it looks like most of the mid-sized dataset competitions are won by Gradient boosting based models. However I came accross case where LightGBM,...
0
votes
1answer
93 views

What is “Missing” in output of plot_tree API of XGBoost

What this "Missing" term means here at each node after split in Image? and also what is at leaf, is this means prediction value? I converted Output variable to 1 and 0. I tried searching on ...
5
votes
1answer
825 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to set ...