Questions tagged [lightgbm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
21
votes
1answer
9k views

Lightgbm vs xgboost vs catboost

I've seen that in Kaggle competitions people are using lightgbms where they used to use xgboost. My question is: when would you rather use xgboost instead of lightgbm? What about catboost?
14
votes
3answers
12k views

L1 & L2 Regularization in Light GBM

This question pertains to L1 & L2 regularization parameters in Light GBM. As per official documentation: reg_alpha (float, optional (default=0.)) – L1 ...
12
votes
4answers
3k views

LightGBM gives different results (metrics) depending on the columns order

I have two nearly identical datasets A and B which differ only in terms of columns ordering. I then train a LightGBM model on each of the two datasets with the following steps: Divide each dataset ...
11
votes
2answers
3k views

What is the proper way to use early stopping with cross-validation?

I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
7
votes
1answer
5k views

Random Forest VS LightGBM

Random Forest VS LightGBM Can somebody explain in-detailed differences between Random Forest and LightGBM? And how the algorithms work under the hood? As per my understanding from the documentation: ...
7
votes
1answer
1k views

Math Behind GOSS (Gradient-Based One Side Sampling)?

As per my understanding through books & Google Search, GOSS (Gradient-Based One Side Sampling) is a novel sampling method that downsamples the instances on the basis of gradients. As we know ...
6
votes
1answer
8k views

Differences between class_weight and scale_pos weight in LightGBM

I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
6
votes
2answers
2k views

SHAP value analysis gives different feature importance on train and test set

Should SHAP value analysis be done on the train or test set? What does it mean if the feature importance based on mean |SHAP value| is different between the train and test set of my lightgbm model? ...
6
votes
1answer
134 views

What is the best way (cheapest / fastest option) to train an model on massive dataset (400GB+, 100m rows x 200 columns)?

I have a 400GB data set that I want to train a model on. What is the cheapest method to train this model? The options I can think of so far are: AWS instance with massive RAM and train CPU (slow, but ...
5
votes
1answer
5k views

How do GBM algorithms handle missing data?

How do algorithms GBM algorithms, such as XGBoost or LightGBM handle NaN values? I know that they learn how to replace NaN values with other values but my question is: How do they do it exactly?
5
votes
1answer
1k views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to set ...
4
votes
4answers
13k views

How to make LightGBM to suppress output?

I have tried for a while to figure out how to "shut up" LightGBM. Especially, I would like to suppress the output of LightGBM during training (the feedback on the boosting steps). My model: <...
4
votes
1answer
2k views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
4
votes
2answers
2k views

Correct interpretation of summary_plot shap graph

While through the various resources online to understand the shap plots, I ended up slightly confused. Find below my interpretation of the overall plot given in examples - Shap value 0 for a feature ...
3
votes
2answers
3k views

Light GBM Regressor, L1 & L2 Regularization and Feature Importances

I want to know how L1 & L2 regularization works in Light GBM and how to interpret the feature importances. Scenario is: I used LGBM Regressor with RandomizedSearchCV (cv=3, iterations=50) on a ...
3
votes
1answer
2k views

How is the "base value" of SHAP values calculated?

I'm trying to understand how the base value is calculated. So I used an example from SHAP's github notebook, Census income classification with LightGBM. Right after I trained the lightgbm model, I ...
3
votes
1answer
958 views

What does the repeated message "No further splits with positive gain, best gain: -inf" mean?

I am training a LightGBM classifier on a binary classification problem. From time to time I get the following message repeatedly: ...
2
votes
1answer
2k views

Lightgbm confidence interval

I want to compute a confidence interval for each sample for a lightgbm model I've trained. If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
2
votes
1answer
2k views

How to use r2-score as a loss function in LightGBM?

I am trying to implement a custom loss function in LightGBM for a regression problem. The intrinsic metrics do not help me much, because they penalise for outliers... Is there any way to use ...
2
votes
1answer
185 views

Extremely high gain with LightGBM

I am working on a binary classification problem. The target variable is not linearly separable, so I've decided to use LightGBM with default parameters (I only play with n_estimators on range from 10 -...
2
votes
1answer
1k views

LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results

While passing the exact same parameters to LightGBM and sklearn's implementation of LightGBM, I am getting different results. Initially, I was getting the exact same results on doing this, however, I ...
2
votes
1answer
27 views

Optimizing MAE degrades MAE metrics

I have run a lighgbm regression model by optimizing on RMSE and measuring the performance on RMSE: ...
2
votes
0answers
189 views

Binary classification with imbalanced dataset, about lightgbm output probability distribution

I trained a binary classifier for an imbalanced dataset. I did two experiments: lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample After training the lgbm model, I ...
2
votes
2answers
34 views

confused on "real score" vs "decision value" in classification trees

I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is: A ...
2
votes
1answer
64 views

How can i tell if my model is overfitting from the distribution of predicted probabilities?

all, i am training light gradient boosting and have used all of the necessary parameters to help in over fitting.i plot the predicted probabilities (i..e probabililty has cancer) distribution from the ...
2
votes
0answers
172 views

Comparing feature importance in LightGBM + Scikit

I have a model trained using LightGBM (LGBMRegressor), in Python, with scikit-learn. On a weekly basis the model in re-trained, and an updated set of chosen features and associated ...
1
vote
2answers
1k views

handling missing values for LightGBM model

I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc.., I have seen some people using ...
1
vote
1answer
37 views

Does Gradient Boosting perform n-ary splits where n > 2?

I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
1
vote
1answer
74 views

Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?

I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
1
vote
1answer
80 views

feature importance and xgboost?

Let say I got feature importance for xgclassifier ...
1
vote
1answer
156 views

Number of leaves for lightgbm is smaller than categories in one feature

I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
1
vote
1answer
707 views

Read back a saved LGBMClassifier model

I trained a LGBMClassifier model and saved in a file in this way: ...
1
vote
1answer
93 views

LightGBM choice of evaluation metric

I have past data of a large number of people who applied for a loan and their movement through 8 different stages, from start of application to loan being paid out. I am trying to build a model that ...
1
vote
0answers
39 views

Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?

I need to implement a custom loss function. The function is relatively simple: $$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$ With $O$ being ...
1
vote
1answer
83 views

Why does Light GBM model produce different results while testing?

Using the Light GBM regressor, I have trained my data and, using Grid Search, I got the best parameters, but while testing with the best parameters I am getting different results each time, which ...
1
vote
0answers
33 views

cost sensitive loss function in lightbm with individual cost

i am looking for a cost sensitive function that will have weights according to individual row feature (like amount) this way i can penalize more FN which has large amount vs. low dollar amount. took ...
1
vote
1answer
157 views

Negative R2_score Bad predictions for my Sales prediction problem using LightGBM

My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows: <...
1
vote
0answers
76 views

Feature Selection before modeling with Boosting Trees

I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
1
vote
1answer
123 views

How to specify scale_pos_weight value at runtime in Hyperopt?

I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown ...
1
vote
0answers
68 views

Does zero_as_missing parameter affects categorical features? LightGBM

Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names. LightGBM documentation says ...
1
vote
0answers
206 views

Gradient boosting Regression with zero-inflated outcome

I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
1
vote
0answers
400 views

Use LightGBM or FFM - imbalanced dataset

I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
1
vote
0answers
484 views

How to save a lightGBM model that updates predictions after each fold?

Hi When I use gradient boosting on Kaggle and large data sets, I run a code like this: ...
1
vote
0answers
482 views

How does L1 Loss work in lightGBM

From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance. My question is that, when the objective is L1 loss/...
0
votes
2answers
32 views

Churn prediction model doesn't predict good on real data

I am working currently on churn prediction problem. As an input I use data from date warehouse for a period 082016 - 032021(one row per month for each customer). Based on this data I have created a ...
0
votes
1answer
45 views

Why does Catboost outperform other boosting algorithms?

I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
0
votes
1answer
37 views

LightGBM regressor score function?

I'm trying to find what is the score function for the LightGBM regressor. In their documentation page I could not find any information regarding the function used to calculate the score attribute...
0
votes
1answer
11 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
0
votes
1answer
199 views

training gradient boosting algorithm in python testing in Golang

What are the best strategy to train and save a gradient boosting algorithm, e.g. LightGBM or XGboost or Catboost in Python but load the model in GoLang and make prediction with Golang ?
0
votes
1answer
34 views

Should we do Feature selection in parallel with feature engineering?

I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i ...