# Tag Info

Accepted

### Am I building a good or bad model for prediction built using Gradient Boosting Classifier Algorithm?

"Unbalanced" data are not a problem, unless you use unsuitable error measures... like accuracy, or precision, recall and the F1 (or any other Fbeta) score, all of which suffer from exactly ...

### Am I building a good or bad model for prediction built using Gradient Boosting Classifier Algorithm?

Your model might be alright, but certainly not with a default classification threshold. As you can see, you only detected 2 out of 123 events this way. ROC_AUC tends to be overoptimistic for this ...
• 656

### Am I building a good or bad model for prediction built using Gradient Boosting Classifier Algorithm?

Your minority class is highly under-represented. I recommend not to proceed forward. My suggestion would be the following: 1.) Undersample the majority class 2.) Use SMOTE to oversample the minority ...
• 109
Accepted

### What is Pruning & Truncation in Decision Trees?

Your understanding is correct. xgboost has nice explanation in the docs. Reading the original papers is always great idea. Here's one for LGBM and here's one for ...
• 814
Accepted

### Reasons for a model predicting probability of being class 1 at x value

Not necessarily, while it can be the case that two observations belong to the same 'group' and end up in the same leaf node (and thus get the same predicted value) there can also be multiple groups of ...
• 6,872
Accepted

### Catboost not able to handle a very simple dataset?

"Traditional" tree models cannot extrapolate well outside the training data's range, so "I want to use shuffle = False for reasons beyond the scope ...
• 10.8k

### What if root of a such tree is pruned in xgboost?

You'll be left with one-node trees. The loss reduction of a split is penalized by $\gamma$, but the root itself does not get pruned. This is fairly easy to test: ...
• 10.8k
Accepted

### XGBoost - Imputing Vs keeping NaN

it is boring and repetitive to say - so sorry - but it depends what are you trying to predict and if it is make sense to do imputation at all. If you are trying to predict the car price probably it is ...
• 313

### XGBoost regression scale invariant? 0 depth trees for target variable with small (1E-7) values

It may not be related to hyperparams per say. I think it has more to do with the nature of how xgboost is trained. XGBoost for regression tries to reduce the variance at every node. May when you have ...
• 1,799
Accepted

### Why is monotonic constraint disabled when using MAE as an objective to LGBM?

MAE is an odd loss function for GBMs: the gradient is constant ($\pm1$), and the hessian all zeros, so the usual tree-training target of $-G/H$ (possibly with additional terms for regularization) ...
• 10.8k
1 vote

### Why is gradient boosting better than random forest for unbalanced data?

Boosting is the method of creating ensemble by increasing the importance of wrongly predicted instances in each iteration. RandomForest works by creating ensemble using Bootstrap Aggregating which ...
• 139
1 vote
Accepted

do they mean that $\sum\limits_{k=1}^{t-1} \omega(f_k) = constant$? Yes! From the paragraph preceding that: we use an additive strategy: fix what we have learned, and add one new tree at a time. ...
• 10.8k
1 vote
Accepted

### How does Catboost regressor deal with categorical features at predict time?

In a simplified way of putting it, we substitute the category id with the mean value of the training set target for this category. CatBoost implements some tricks like only using the preceding values ...
• 656
1 vote

### GridSearch multiplying the number of trees in XGboost?

You have three classes; xgboost is building three one-vs-rest models, hence three times the trees. https://github.com/dmlc/xgboost/issues/806
• 10.8k
1 vote
Accepted

### How is the probability prediction of a binary classifier predicted

y is beta0 + beta1*x1 + beta2*x2 .....+ epsilong probability = ((e power y)/(1 + e power y)) ...
1 vote

### Feature importance by random forest and boosting tree when two features are heavy correlated

Randomized sparse models: Random forest and boosting tree already have the feature (column) sampling, are two effects repeated? This question is unclear. Random Forest and GBDT are not randomized ...
• 5,776
1 vote
Accepted

### If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree? It's harder yes, but common. In the same tree it can not happen in the ...
• 5,776
1 vote

### Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?

In case you hadn't seen the User Guide section for this method, the explanation there is pretty good: These fast estimators first bin the input samples X into ...
• 10.8k
1 vote
Accepted

### Tree complexity and gamma parameter in xgboost

In the paragraph following equation (1): $T$ is the number of leaves in the tree. $\gamma$ is a hyperparameter that affects how much regularization occurs on the size (number of leaves) of the tree. ...
• 10.8k
1 vote

### Chossing between gradient boosting algorithms

I would say Catboost and lightgbm perform similarly and its purely a matter of choice. Some of my colleagues prefered ...
1 vote

### What is "Missing" in output of plot_tree API of XGBoost

The "missing" at the different nodes are the observations for which the feature on which the split is made is missing. E.g. if the value for ...
• 6,872

Only top scored, non community-wiki answers of a minimum length are eligible