Questions tagged [lightgbm]
The lightgbm tag has no usage guidance.
38
questions
1
vote
1answer
62 views
Lightgbm confidence interval
I want to compute a confidence interval for each sample for a lightgbm model I've trained.
If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
0
votes
0answers
27 views
XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?
We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
1
vote
0answers
21 views
Does zero_as_missing parameter affects categorical features? LightGBM
Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names.
LightGBM documentation says ...
6
votes
2answers
377 views
What is the proper way to use early stopping with cross-validation?
I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
0
votes
1answer
21 views
Should we do Feature selection in parallel with feature engineering?
I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i ...
2
votes
1answer
40 views
How is the “base value” of SHAP values calculated?
I'm trying to understand how the base value is calculated. So I used an example from SHAP's github notebook, https://github.com/slundberg/shap/blob/master/notebooks/tree_explainer/Census%20income%...
0
votes
1answer
131 views
How to use r2-score as a loss function in LightGBM?
I am trying to implement a custom loss function in LightGBM for a regression problem. The intrinsic metrics do not help me much, because they penalise for outliers... Is there any way to use ...
1
vote
0answers
63 views
Gradient boosting Regression with zero-inflated outcome
I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
3
votes
1answer
128 views
Small number of estimators in gradient boosting
I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
1
vote
1answer
35 views
-1
votes
1answer
35 views
random forest and log_loss metric?
Light gbm has the metric with log_loss for binary or multi classification.
is Random Forest also has the loss function with log_loss?
0
votes
1answer
17 views
Number of leaves for lightgbm is smaller than categories in one feature
I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
2
votes
0answers
76 views
What does the repeated message “No further splits with positive gain, best gain: -inf” mean?
I am training a LightGBM classifier on a binary classification problem.
From time to time I get the following message repeatedly:
...
4
votes
1answer
791 views
How do GBM algorithms handle missing data?
How do algorithms GBM algorithms, such as XGBoost or LightGBM handle NaN values?
I know that they learn how to replace NaN values with other values but my question is: How do they do it exactly?
2
votes
1answer
46 views
Correct interpretation of summary_plot shap graph
While through the various resources online to understand the shap plots, I ended up slightly confused. Find below my interpretation of the overall plot given in examples -
Shap value 0 for a feature ...
2
votes
1answer
90 views
Extremely high gain with LightGBM
I am working on a binary classification problem. The target variable is not linearly separable, so I've decided to use LightGBM with default parameters (I only play with n_estimators on range from 10 -...
1
vote
1answer
301 views
Math Behind GOSS (Gradient-Based One Side Sampling)?
As per my understanding through books & Google Search,
GOSS (Gradient-Based One Side Sampling) is a novel sampling method that downsamples the instances on the basis of gradients. As we know ...
5
votes
1answer
238 views
What is Pruning & Truncation in Decision Trees?
Pruning & Truncation
As per my understanding
Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to set ...
2
votes
1answer
1k views
Random Forest VS LightGBM
Random Forest VS LightGBM
Can somebody explain in-detailed differences between Random Forest and LightGBM? And how the algorithms work under the hood?
As per my understanding from the documentation:
...
0
votes
0answers
32 views
Should the LightGBM score match the regularization?
If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
-1
votes
2answers
632 views
1
vote
0answers
173 views
Read back a saved LGBMClassifier model
I trained a LGBMClassifier model and saved in a file in this way:
...
2
votes
1answer
671 views
LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results
While passing the exact same parameters to LightGBM and sklearn's implementation of LightGBM, I am getting different results. Initially, I was getting the exact same results on doing this, however, I ...
4
votes
2answers
251 views
SHAP value analysis gives different feature importance on train and test set
Should SHAP value analysis be done on the train or test set?
What does it mean if the feature importance based on mean |SHAP value| is different between the train and test set of my lightgbm model?
...
0
votes
0answers
29 views
if I got feature importance of xgboost/LightGBM what is next?
If I have feature importances of different variables in a xgboost/LightGBM model, how do I use this information?
Is it better to just use the top n features and retrain the model?
Does the feature ...
1
vote
0answers
224 views
Use LightGBM or FFM - imbalanced dataset
I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
1
vote
1answer
43 views
LightGBM choice of evaluation metric
I have past data of a large number of people who applied for a loan and their movement through 8 different stages, from start of application to loan being paid out.
I am trying to build a model that ...
0
votes
1answer
94 views
Sliding window approach using SVR & LightGBM
I'm working on a multivariate time series forecast using a couple of ML algorithms (Neural Networks, Support Vector Machines & Gradient boosting algorithms). I need to measure the performance of ...
0
votes
1answer
474 views
How to choose the model parameters (RandomizedSearchCV, .GridSearchCV) or manually
Faced with the task of selecting parameters for the lightgbm model, the question accordingly arises, what is the best way to select them? I used the RandomizedSearchCV method, within 10 hours the ...
2
votes
0answers
86 views
Comparing feature importance in LightGBM + Scikit
I have a model trained using LightGBM (LGBMRegressor), in Python, with scikit-learn.
On a weekly basis the model in re-trained, and an updated set of chosen features and associated ...
9
votes
2answers
4k views
L1 & L2 Regularization in Light GBM
This question pertains to L1 & L2 regularization parameters in Light GBM. As per official documentation:
reg_alpha (float, optional (default=0.)) ā L1 ...
3
votes
1answer
858 views
Light GBM Regressor, L1 & L2 Regularization and Feature Importances
I want to know how L1 & L2 regularization works in Light GBM and how to interpret the feature importances.
Scenario is: I used LGBM Regressor with RandomizedSearchCV (cv=3, iterations=50) on a ...
0
votes
1answer
284 views
Hyperparameter Tuning Time Series in Production
I have a time series data that handled using GDBT to predict the next value. I always use previous 30 days data to train daily, but overtime the data to predict and train is increased because the ...
1
vote
0answers
274 views
How to save a lightGBM model that updates predictions after each fold?
Hi When I use gradient boosting on Kaggle and large data sets, I run a code like this:
...
0
votes
1answer
27 views
Syntax of the inputs for LightGBM binary build (without python API)
I built the LightGBM binaries from https://github.com/microsoft/LightGBM. The build was successful, but the python API did not work for me. I am using the light gbm binary directly to run for time ...
3
votes
1answer
2k views
Differences between class_weight and scale_pos weight in LightGBM
I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
0
votes
2answers
4k views
How to make LightGBM to suppress output?
I'm trying for a while to figure out how to "shut up" LightGBM. Especially, I want to suppress the output of LightGBM during training (the feedback on the boosting steps).
My model:
...
1
vote
0answers
206 views
How does L1 Loss work in lightGBM
From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance.
My question is that, when the objective is L1 loss/...
