Questions tagged [lightgbm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
16 views

How can I implement lambda-mart with lightgbm?

I have a learning to rank task at hand and I want to use the lightgbm implementation of LambdaMART. I'm also following this notebook. ...
user avatar
0 votes
0 answers
9 views

How does exactly eval_set and RandomizedSearchCV work for LightGBM?

How does RandomizedSearchCV form the validation sets, while I also defined an evaluation set for LGBM? Is it formed from the train set I gave or how does the evaluation set comes into the validation? ...
user avatar
0 votes
0 answers
10 views

Constructing pandas DF with model trained categorical variable

I've trained a lightGBM model on a dataset X where X has a categorical (in the pandas sense) variable. This model trains fine and when I predict using it all looks good - I can even change the value ...
user avatar
0 votes
1 answer
11 views

SOS: Working LightGBM model script to find best model

I have been trying to get a working LightGBM model which I can train on my data, select the best performing model with highest f1 score and then use it obtain the f1 score on the testing data. However,...
user avatar
  • 1
0 votes
0 answers
21 views

Does lightGBM handle multicollinearity? [duplicate]

I have a dataset after feature selection of around 6500 features and 10,000 data rows. I am using LightGBM model. I want to know if I should check the feature set for multicollinearity. If two or more ...
user avatar
  • 27
0 votes
0 answers
15 views

splitting point in LightGBM?

I am not able to understand how the first root node is selected in LightGBM and how the splitting at nodes happens further. I read blogs and related documents and I understand that in this histogram-...
user avatar
  • 27
1 vote
1 answer
28 views

Model Performance on external validation Set really low?

I am using the LGBM model for binary classification. My train and test accuracies are 87% & 82% respectively with cross-validation of 89%. ROC-AUC score of 81%. But when evaluating model ...
user avatar
  • 27
0 votes
0 answers
15 views

LGBM model predicting only single class on unseen data!

I have built a LightGBM based machine learning model on data of molecules of two classes. The distribution is as follows. Class 0 has 5933 data points and class 1 has 4696. The train test accuracy I ...
user avatar
  • 27
0 votes
3 answers
56 views

Is my model overfitting ? Training Acc :93 % test accuracy 82%

I am using LGBM model for binary classification. After hyper-parameter tuning I get Training accuracy 0.9340 Test accuracy 0.8213 can I say my model is overfitting?...
user avatar
  • 27
0 votes
0 answers
42 views

LightGBM predict_proba in thousandths place

Can someone explain to me how my lightgbm classification model's predict_proba() is in thousandths place for the positive class: ...
user avatar
  • 101
1 vote
1 answer
48 views

Random LightGBM Forest

I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
user avatar
0 votes
1 answer
26 views

Proof of GOSS algorithm in lightGBM paper

In the LightGBM paper the authors make use of a newly developed sampling method GOSS to reduce the number of data instances needed for finding the best split of a ...
user avatar
  • 101
0 votes
1 answer
24 views

Incorporating data over time into lightgbm

So I'm in the situation where I know what it is I'm trying to find, but not the terminology for it and I think that's why a lot of my google searches are directing me in the wrong direction, so ...
user avatar
0 votes
1 answer
288 views

LightGBM eval_set - what to do when I fit the final model (there's no test data left)

I'm using LightGBM's eval_set feature when fitting my model. This enables early stopping on the number of estimators used. ...
user avatar
1 vote
1 answer
48 views

Understanding feature_parallel distributed learning algorithm in LightGBMClassifier

I want to understand feature_parallel algorithm in LightGBMClassifier. It describes how it is done traditionally and how LightGBM...
user avatar
2 votes
3 answers
58 views

Example for Boosting

Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
user avatar
  • 173
1 vote
0 answers
12 views

Fractional Differencing/Differentiation for Non-Time based Model; Look-ahead bias?

I have time-series data, but instead of using a time-based model like RNN, I've decided to approach my classification problem using an lgbm classifier. To do so, I have modified the data, such that ...
user avatar
2 votes
1 answer
70 views

Optimizing MAE degrades MAE metrics

I have run a lighgbm regression model by optimizing on RMSE and measuring the performance on RMSE: ...
user avatar
  • 121
0 votes
1 answer
285 views

LightGBM regressor score function?

I'm trying to find what is the score function for the LightGBM regressor. In their documentation page I could not find any information regarding the function used to calculate the score attribute...
user avatar
0 votes
0 answers
25 views

what is the meaning of split maximum return leaf, non split maximum return leaf in lightgbm

I am currently working on lightgbm, where i read this sentence i am unable to understand what this means. can anyone help me that what it means. thank you. https://www.hindawi.com/journals/mpe/2020/...
user avatar
  • 1
0 votes
2 answers
56 views

Churn prediction model doesn't predict good on real data

I am working currently on churn prediction problem. As an input I use data from date warehouse for a period 082016 - 032021(one row per month for each customer). Based on this data I have created a ...
user avatar
  • 13
0 votes
0 answers
55 views

For a Multi-class Classification Problem, What are the Pros and Cons of using a Cascade ML model versus Single Multi-class Classification model?

I am developing an ML model for classification using tabular data. It has 5 classes right now and new classes are expected to be continuously added. (Already have a new one leading to an imbalanced ...
user avatar
  • 87
0 votes
0 answers
22 views

What is the best way to create input data samples using in XGBoost for predicting number of next days that customer will come back to store

I'm building the tree-based model like a XGBoost to solve the problem about customer purchase cycle. And I think, I will build 2 models which one is predicting the customer will come back to store in ...
user avatar
  • 43
0 votes
1 answer
17 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
user avatar
  • 165
2 votes
0 answers
661 views

Binary classification with imbalanced dataset, about lightgbm output probability distribution

I trained a binary classifier for an imbalanced dataset. I did two experiments: lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample After training the lgbm model, I ...
user avatar
2 votes
2 answers
43 views

confused on "real score" vs "decision value" in classification trees

I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is: A ...
user avatar
  • 121
0 votes
0 answers
16 views

Model variance increases during training

I trained a regression model with lightgbm and the learning curve doesn't look good: The model variance increases during training, which shows a kind of overfitting. Now, I tried many ways to fix ...
user avatar
  • 121
0 votes
0 answers
57 views

LightGBM boosting and bagging parameters

When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg. ...
user avatar
1 vote
0 answers
52 views

Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?

I need to implement a custom loss function. The function is relatively simple: $$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$ With $O$ being ...
user avatar
0 votes
1 answer
27 views

Scikit-learn estimator not changing predictions when random_state variable changes

I am trying to compute prediction intervals for a classifier I trained in scikit-learn. Even after setting a new random_state parameter in my pipeline, this does ...
user avatar
0 votes
0 answers
17 views

LGBM permutation invariance

I'm trying to train a binary classifier with LGBM over pairs of points in $\mathbb{R}^d$ (i.e. a classifier that takes a pair in $\mathbb{R}^{2d}$ and returns a class probability for the pair). I ...
user avatar
0 votes
1 answer
270 views

Why does Light GBM model produce different results while testing?

Using the Light GBM regressor, I have trained my data and, using Grid Search, I got the best parameters, but while testing with the best parameters I am getting different results each time, which ...
user avatar
1 vote
1 answer
320 views

Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?

I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
user avatar
  • 35
0 votes
1 answer
28 views

Random Forest but keep only leaves with impurities below a threshold

Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine? In other words, if I set min ...
user avatar
5 votes
2 answers
195 views

What is the best way (cheapest / fastest option) to train an model on massive dataset (400GB+, 100m rows x 200 columns)?

I have a 400GB data set that I want to train a model on. What is the cheapest method to train this model? The options I can think of so far are: AWS instance with massive RAM and train CPU (slow, but ...
user avatar
  • 181
1 vote
0 answers
46 views

cost sensitive loss function in lightbm with individual cost

i am looking for a cost sensitive function that will have weights according to individual row feature (like amount) this way i can penalize more FN which has large amount vs. low dollar amount. took ...
user avatar
0 votes
1 answer
133 views

Model Dump Parser (like XGBFI) for LightGBM and CatBoost

Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
user avatar
  • 1,517
0 votes
1 answer
55 views

Why does Catboost outperform other boosting algorithms?

I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
user avatar
0 votes
1 answer
127 views

Hyperparameter tuning with Bayesian-Optimization

I'm using LightGBM for the regression problem and here is my code. ...
user avatar
  • 3
1 vote
1 answer
79 views

Does Gradient Boosting perform n-ary splits where n > 2?

I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
user avatar
1 vote
1 answer
193 views

Negative R2_score Bad predictions for my Sales prediction problem using LightGBM

My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows: <...
user avatar
1 vote
0 answers
105 views

Feature Selection before modeling with Boosting Trees

I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
user avatar
  • 11
1 vote
1 answer
202 views

How to specify scale_pos_weight value at runtime in Hyperopt?

I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown ...
user avatar
  • 11
0 votes
1 answer
270 views

training gradient boosting algorithm in python testing in Golang

What are the best strategy to train and save a gradient boosting algorithm, e.g. LightGBM or XGboost or Catboost in Python but load the model in GoLang and make prediction with Golang ?
user avatar
1 vote
2 answers
2k views

handling missing values for LightGBM model

I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc.., I have seen some people using ...
user avatar
0 votes
1 answer
378 views

Correct theoretical regularized objective function for XGB/LGBM (regression task)

I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
user avatar
  • 21
2 votes
1 answer
77 views

How can i tell if my model is overfitting from the distribution of predicted probabilities?

all, i am training light gradient boosting and have used all of the necessary parameters to help in over fitting.i plot the predicted probabilities (i..e probabililty has cancer) distribution from the ...
user avatar
  • 466
2 votes
1 answer
2k views

Lightgbm confidence interval

I want to compute a confidence interval for each sample for a lightgbm model I've trained. If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
user avatar
  • 5,654
1 vote
0 answers
93 views

Does zero_as_missing parameter affects categorical features? LightGBM

Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names. LightGBM documentation says ...
user avatar
12 votes
2 answers
4k views

What is the proper way to use early stopping with cross-validation?

I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
user avatar