Questions tagged [gbm]
Gradient Boosting Machine
18
questions with no upvoted or accepted answers
9
votes
0
answers
2k
views
AdaBoost implementation and tuning for high dimensional feature space in R
I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
- 91
2
votes
1
answer
208
views
Why is each successive tree in GBM fit on the negative gradient of the loss function?
Page 359 of Elements Of Statistical Learning 2nd edition says the below.
Can someone explain the intuition & simplify it in layman terms?
Questions
What is the reason/intuition & math ...
- 2,028
1
vote
0
answers
36
views
Which loss functions does h2o.gbm use by default?
the GBM implementation of the h2o package only allows the user to specify a loss function via the distribution argument, which defaults to ...
- 11
1
vote
0
answers
76
views
How to use "related" and "unrelated" as classes rather than multiple classes?
I have a dataset with about 15 feature columns and about 1000 rows that I'd like to use for supervised training.
Every row can be classified as "related" or "unrelated" to another row. About fifteen ...
1
vote
1
answer
206
views
GBM: small change in the trainset causes radical change in predictions
I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
1
vote
0
answers
354
views
using word embedding features with linear prediction models
I have been seeing that word embedding features (e.g. here or there) are used on classification or regression tasks where the classifier/regressor is a linear one: e.g. Linear/Logistic Regressor or ...
- 141
1
vote
0
answers
1k
views
Bayesian optimization for a Light GBM Model
I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
- 11
1
vote
0
answers
967
views
How does L1 Loss work in lightGBM
From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance.
My question is that, when the objective is L1 loss/...
1
vote
1
answer
71
views
Is this random forest logical correct and correct implemented with R and gbm?
For professional reasons I want to learn and understand random forests. I feel unsafe if my understanding is the correct or if I am doing logical errors.
I got a data set with 15 million entries and ...
- 31
1
vote
0
answers
70
views
Why is my predicted vs observed plot worse for training than validation. Running an overfitted GBM on a binomial outcome
I have a binomial outcome that I am trying to predict using a gbm in h2o.
I have set quite a low min_rows value for each node and it appears to be overfitting. See plots below.
When I group the ...
- 11
1
vote
0
answers
98
views
Injecting random values as one input feature for feature selection results in a odd beaviour
I am trying to find a cutoff value, in the feature importance space to eliminate spurious features. So I am injecting a completely random generated feature (as one of the input features) and I cut the ...
- 415
1
vote
0
answers
29
views
Convert Out-of-bag (OOB) estimate to quad weighted kappa score
Is there a way to directly calculate an approximate quad weighted kappa measure from an OOB estimate, obtained from a gradient boosting model with subsampling without going through cross validation?
- 151
1
vote
0
answers
316
views
Unbalanced data fit in gbm
I'm trying to build a model using GBM in r in order to get probability of two classes ( 'yes','no'). My data are unbalanced, and because of this I trained my model using a balanced data(undersampling ...
- 83
0
votes
0
answers
21
views
Question on theory from original GBM article
I am reading the original gradient boosting machine article and, maybe because my statistics are a bit rusty, have a few questions on one section.
In section ...
- 11
0
votes
0
answers
13
views
Why are we replacing values in leaves in gradient boosting?
As far as I understand, we decompose the error function into a Taylor series, find its derivative, set it to zero and get a new optimal final value for the sheet. This is the gradient step in the ...
- 11
0
votes
0
answers
16
views
What is the complexity of ICE (Individual conditional expectation) on ensemble trees
I'm evaluating different model-agnostic methods on gradient boosting or random forest model.
For Shapley, specifically TreeSHAP, the complexity is O(TLD^2) according to Lundberg et al. 2018. T: #trees,...
- 1
0
votes
2
answers
386
views
aggregation of feature importance
I have more of a conceptual question I was hoping to get some feedback on. I am trying to run a boosted regression ML model to identify a subset of important predictors for some clinical condition. ...
- 103
0
votes
1
answer
1k
views
Why does Light GBM model produce different results while testing?
Using the Light GBM regressor, I have trained my data and, using Grid Search, I got the best parameters, but while testing with the best parameters I am getting different results each time, which ...
