Questions tagged [lightgbm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
43 views

Model Dump Parser (like XGBFI) for LightGBM and CatBoost

Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
1
vote
1answer
85 views

Negative R2_score Bad predictions for my Sales prediction problem using LightGBM

My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows: <...
0
votes
0answers
8 views

LightGBM model improvement when the focus is on probability prediction

I am building a binary classifier using LightGBM. The goal is not to predict the outcome as such, but rather to predict the probability of the target even. To be more specific, it's more about ranking ...
1
vote
0answers
41 views

Feature Selection before modeling with Boosting Trees

I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
0
votes
0answers
21 views

LightGBM get model decision(rules)

I need to interpret the model decision for binary classification. Here is my model: ...
1
vote
1answer
34 views

How to specify scale_pos_weight value at runtime in Hyperopt?

I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown ...
6
votes
1answer
97 views

What is the best way (cheapest / fastest option) to train an model on massive dataset (400GB+, 100m rows x 200 columns)?

I have a 400GB data set that I want to train a model on. What is the cheapest method to train this model? The options I can think of so far are: AWS instance with massive RAM and train CPU (slow, but ...
0
votes
0answers
6 views

Tree Based Classification (XGBoost, LightGBM, etc) - Features from embeddings for sparse features?

I'm wondering if there is a possibility from using embeddings as inputs for tree based classification models? For example we have a field called type of food, and ...
1
vote
0answers
21 views

cost sensitive loss function in lightbm with individual cost

i am looking for a cost sensitive function that will have weights according to individual row feature (like amount) this way i can penalize more FN which has large amount vs. low dollar amount. took ...
5
votes
1answer
805 views

Math Behind GOSS (Gradient-Based One Side Sampling)?

As per my understanding through books & Google Search, GOSS (Gradient-Based One Side Sampling) is a novel sampling method that downsamples the instances on the basis of gradients. As we know ...
0
votes
0answers
16 views

Why does Catboost outperform other boosting algorithms?

I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
3
votes
1answer
503 views

What does the repeated message “No further splits with positive gain, best gain: -inf” mean?

I am training a LightGBM classifier on a binary classification problem. From time to time I get the following message repeatedly: ...
0
votes
1answer
136 views

Sliding window approach using SVR & LightGBM

I'm working on a multivariate time series forecast using a couple of ML algorithms (Neural Networks, Support Vector Machines & Gradient boosting algorithms). I need to measure the performance of ...
0
votes
1answer
28 views

Hyperparameter tuning with Bayesian-Optimization

I'm using LightGBM for the regression problem and here is my code. ...
5
votes
1answer
645 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to set ...
5
votes
1answer
4k views

Differences between class_weight and scale_pos weight in LightGBM

I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
2
votes
2answers
705 views

Correct interpretation of summary_plot shap graph

While through the various resources online to understand the shap plots, I ended up slightly confused. Find below my interpretation of the overall plot given in examples - Shap value 0 for a feature ...
0
votes
1answer
182 views

Correct theoretical regularized objective function for XGB/LGBM (regression task)

I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
0
votes
0answers
13 views

Gbm prediction distribution different to training data

I’m doing a regression on a dataset using lightgbm. For the training data the response variable has a non normal distribution which is multimodal. However, the predictions out-of-fold have are ...
1
vote
1answer
24 views

Does Gradient Boosting perform n-ary splits where n > 2?

I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
0
votes
0answers
92 views

how to choose importance_type in lgbm.plot_importance

I have made a binary-classifier using lgbm.The classifier is made on unbalanced dataset. I wanted to see the importance features of the model. There are two types of selecting importance_type - ...
3
votes
1answer
1k views

How is the “base value” of SHAP values calculated?

I'm trying to understand how the base value is calculated. So I used an example from SHAP's github notebook, Census income classification with LightGBM. Right after I trained the lightgbm model, I ...
0
votes
1answer
98 views

training gradient boosting algorithm in python testing in Golang

What are the best strategy to train and save a gradient boosting algorithm, e.g. LightGBM or XGboost or Catboost in Python but load the model in GoLang and make prediction with Golang ?
0
votes
0answers
163 views

Understanding interplay between eval_metric, metric and first_metric_only parameters in LGBMClassifier

In python API of LGBMClassifier, the constructor takes parameters metric and first_metric_only. Their descriptions are as ...
0
votes
2answers
386 views

handling missing values for LightGBM model

I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc.., I have seen some people using ...
2
votes
1answer
58 views

How can i tell if my model is overfitting from the distribution of predicted probabilities?

all, i am training light gradient boosting and have used all of the necessary parameters to help in over fitting.i plot the predicted probabilities (i..e probabililty has cancer) distribution from the ...
3
votes
2answers
2k views

Light GBM Regressor, L1 & L2 Regularization and Feature Importances

I want to know how L1 & L2 regularization works in Light GBM and how to interpret the feature importances. Scenario is: I used LGBM Regressor with RandomizedSearchCV (cv=3, iterations=50) on a ...
2
votes
1answer
1k views

LightGBM vs Sklearn LightGBM- Mistake in Implementation- Exact same parameters giving different results

While passing the exact same parameters to LightGBM and sklearn's implementation of LightGBM, I am getting different results. Initially, I was getting the exact same results on doing this, however, I ...
1
vote
1answer
99 views

Number of leaves for lightgbm is smaller than categories in one feature

I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
5
votes
2answers
936 views

SHAP value analysis gives different feature importance on train and test set

Should SHAP value analysis be done on the train or test set? What does it mean if the feature importance based on mean |SHAP value| is different between the train and test set of my lightgbm model? ...
4
votes
1answer
3k views

How do GBM algorithms handle missing data?

How do algorithms GBM algorithms, such as XGBoost or LightGBM handle NaN values? I know that they learn how to replace NaN values with other values but my question is: How do they do it exactly?
1
vote
1answer
905 views

Lightgbm confidence interval

I want to compute a confidence interval for each sample for a lightgbm model I've trained. If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
0
votes
0answers
35 views

XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?

We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
1
vote
0answers
41 views

Does zero_as_missing parameter affects categorical features? LightGBM

Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names. LightGBM documentation says ...
7
votes
2answers
1k views

What is the proper way to use early stopping with cross-validation?

I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
0
votes
1answer
29 views

Should we do Feature selection in parallel with feature engineering?

I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i ...
2
votes
2answers
8k views

How to make LightGBM to suppress output?

I'm trying for a while to figure out how to "shut up" LightGBM. Especially, I want to suppress the output of LightGBM during training (the feedback on the boosting steps). My model: ...
2
votes
1answer
868 views

How to use r2-score as a loss function in LightGBM?

I am trying to implement a custom loss function in LightGBM for a regression problem. The intrinsic metrics do not help me much, because they penalise for outliers... Is there any way to use ...
1
vote
0answers
150 views

Gradient boosting Regression with zero-inflated outcome

I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
3
votes
1answer
707 views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
1
vote
1answer
52 views

feature importance and xgboost?

Let say I got feature importance for xgclassifier ...
0
votes
1answer
518 views

random forest and log_loss metric?

Light gbm has the metric with log_loss for binary or multi classification. is Random Forest also has the loss function with log_loss?
2
votes
1answer
143 views

Extremely high gain with LightGBM

I am working on a binary classification problem. The target variable is not linearly separable, so I've decided to use LightGBM with default parameters (I only play with n_estimators on range from 10 -...
6
votes
1answer
4k views

Random Forest VS LightGBM

Random Forest VS LightGBM Can somebody explain in-detailed differences between Random Forest and LightGBM? And how the algorithms work under the hood? As per my understanding from the documentation: ...
-1
votes
2answers
1k views
1
vote
0answers
487 views

Read back a saved LGBMClassifier model

I trained a LGBMClassifier model and saved in a file in this way: ...
1
vote
0answers
329 views

Use LightGBM or FFM - imbalanced dataset

I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
1
vote
1answer
63 views

LightGBM choice of evaluation metric

I have past data of a large number of people who applied for a loan and their movement through 8 different stages, from start of application to loan being paid out. I am trying to build a model that ...
0
votes
1answer
1k views

How to choose the model parameters (RandomizedSearchCV, .GridSearchCV) or manually

Faced with the task of selecting parameters for the lightgbm model, the question accordingly arises, what is the best way to select them? I used the RandomizedSearchCV method, within 10 hours the ...
2
votes
0answers
151 views

Comparing feature importance in LightGBM + Scikit

I have a model trained using LightGBM (LGBMRegressor), in Python, with scikit-learn. On a weekly basis the model in re-trained, and an updated set of chosen features and associated ...