Questions tagged [natural-gradient-boosting]

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Uncertainty prediction in Gradient Boosted Tree based Quantile Regression

For an application, I am using a Gradient boosting Tree based quantile regression model (LightGBM, Catboot) to predict the 5th percentile of the target variable. The model predicts point estimates, ...
0
votes
0answers
20 views

When to use Gradient boosting over stochastic gradient boosting

Gradient boosting works on the Gradient Descent concept and it's one of the ensemble methods. It has a regularization parameter to select subsamples, which is called stochastic gradient boosting. ...
2
votes
1answer
40 views

House price inflation modelling

I have a data set of house prices and their corresponding features (rooms, meter squared, etc). An additional feature is the sold date of the house. The aim is to create a model that can estimate the ...
0
votes
1answer
104 views

How to reconstruct a scikit-learn predictor for Gradient Boosting Regressor?

I would like to train my datasets in scikit-learn but export the final Gradient Boosting Regressor elsewhere so that I can make predictions directly on another ...
1
vote
1answer
43 views

Does Gradient Boosting perform n-ary splits where n > 2?

I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
2
votes
1answer
72 views

DecisionTreeRegressor under the hood of GradientBoostingClassifier

I'm inspecting the weak estimators of my GradientBoostingClassifier model. This model was fit on a binary class dataset. I noticed that all the weak estimators under this ensemble classifier are ...
0
votes
1answer
119 views

Which other algorithms fit residuals like XGBoost? [closed]

XGBoost and standard gradient boosting train learners to fit the residuals rather than the observations themselves. I understand that this aspect of the algorithm matches the boosting mechanism which ...
2
votes
1answer
38 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set $\{(x_{i},y_{i})\}_{i=1}^{n}$, a differentiable loss function $L(y,F(x))$, and a number of ...
9
votes
4answers
2k views

Can Boosted Trees predict below the minimum value of the training label?

I am using gradient Gradient Boosted Trees (with Catboost) for a Regression task. Can GBtrees predict a label that is below the minimum (or above the max) that was seen in the training ? For instance ...
3
votes
1answer
210 views

Why would GradientBoostClassifier do better than XGBoostClassifier? [duplicate]

I am working on the Kaggle home loan model and interestingly enough, the GradientBoostClassifier has a considerably better score than XGBClassifier. At the same time it seems to not overfit as much. (...
2
votes
1answer
439 views

How to enable GPU on GradientBoostingClassifier?

Is there a way to enable GPU on GradientBoostingClassifier?
0
votes
1answer
261 views

Handling Categorical Features on NGBoost

Recently I have been doing some research on NGBoost, but I could not see any parameter for categorical features. Is there any parameter that I missed? ...