Questions tagged [lightgbm]
The lightgbm tag has no usage guidance.
84
questions
1
vote
1
answer
16
views
How can I implement lambda-mart with lightgbm?
I have a learning to rank task at hand and I want to use the lightgbm implementation of LambdaMART. I'm also following this notebook.
...
- 664
0
votes
0
answers
9
views
How does exactly eval_set and RandomizedSearchCV work for LightGBM?
How does RandomizedSearchCV form the validation sets, while I also defined an evaluation set for LGBM? Is it formed from the train set I gave or how does the evaluation set comes into the validation?
...
0
votes
0
answers
10
views
Constructing pandas DF with model trained categorical variable
I've trained a lightGBM model on a dataset X where X has a categorical (in the pandas sense) variable. This model trains fine and when I predict using it all looks good - I can even change the value ...
0
votes
1
answer
11
views
SOS: Working LightGBM model script to find best model
I have been trying to get a working LightGBM model which I can train on my data, select the best performing model with highest f1 score and then use it obtain the f1 score on the testing data. However,...
- 1
0
votes
0
answers
21
views
Does lightGBM handle multicollinearity? [duplicate]
I have a dataset after feature selection of around 6500 features and 10,000 data rows. I am using LightGBM model. I want to know if I should check the feature set for multicollinearity. If two or more ...
- 27
0
votes
0
answers
15
views
splitting point in LightGBM?
I am not able to understand how the first root node is selected in LightGBM and how the splitting at nodes happens further. I read blogs and related documents and I understand that in this histogram-...
- 27
1
vote
1
answer
28
views
Model Performance on external validation Set really low?
I am using the LGBM model for binary classification. My train and test accuracies are 87% & 82% respectively with cross-validation of 89%. ROC-AUC score of 81%. But when evaluating model ...
- 27
0
votes
0
answers
15
views
LGBM model predicting only single class on unseen data!
I have built a LightGBM based machine learning model on data of molecules of two classes. The distribution is as follows. Class 0 has 5933 data points and class 1 has 4696. The train test accuracy I ...
- 27
0
votes
3
answers
56
views
Is my model overfitting ? Training Acc :93 % test accuracy 82%
I am using LGBM model for binary classification. After hyper-parameter tuning I get
Training accuracy 0.9340
Test accuracy 0.8213
can I say my model is overfitting?...
- 27
0
votes
0
answers
42
views
LightGBM predict_proba in thousandths place
Can someone explain to me how my lightgbm classification model's predict_proba() is in thousandths place for the positive class:
...
- 101
1
vote
1
answer
48
views
Random LightGBM Forest
I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
- 358
0
votes
1
answer
26
views
Proof of GOSS algorithm in lightGBM paper
In the LightGBM paper the authors make use of a newly developed sampling method GOSS to reduce the number of data instances needed for finding the best split of a ...
- 101
0
votes
1
answer
24
views
Incorporating data over time into lightgbm
So I'm in the situation where I know what it is I'm trying to find, but not the terminology for it and I think that's why a lot of my google searches are directing me in the wrong direction, so ...
- 151
0
votes
1
answer
288
views
LightGBM eval_set - what to do when I fit the final model (there's no test data left)
I'm using LightGBM's eval_set feature when fitting my model. This enables early stopping on the number of estimators used.
...
- 235
1
vote
1
answer
48
views
Understanding feature_parallel distributed learning algorithm in LightGBMClassifier
I want to understand feature_parallel algorithm in LightGBMClassifier.
It describes how it is done traditionally and how LightGBM...
- 575
2
votes
3
answers
58
views
Example for Boosting
Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
- 173
1
vote
0
answers
12
views
Fractional Differencing/Differentiation for Non-Time based Model; Look-ahead bias?
I have time-series data, but instead of using a time-based model like RNN, I've decided to approach my classification problem using an lgbm classifier. To do so, I have modified the data, such that ...
- 11
2
votes
1
answer
70
views
Optimizing MAE degrades MAE metrics
I have run a lighgbm regression model by optimizing on RMSE and measuring the performance on RMSE:
...
- 121
0
votes
1
answer
285
views
LightGBM regressor score function?
I'm trying to find what is the score function for the LightGBM regressor. In their documentation page I could not find any information regarding the function used to calculate the score attribute...
0
votes
0
answers
25
views
what is the meaning of split maximum return leaf, non split maximum return leaf in lightgbm
I am currently working on lightgbm, where i read this sentence i am unable to understand what this means. can anyone help me that what it means.
thank you.
https://www.hindawi.com/journals/mpe/2020/...
- 1
0
votes
2
answers
56
views
Churn prediction model doesn't predict good on real data
I am working currently on churn prediction problem.
As an input I use data from date warehouse for a period 082016 - 032021(one row per month for each customer).
Based on this data I have created a ...
- 13
0
votes
0
answers
55
views
For a Multi-class Classification Problem, What are the Pros and Cons of using a Cascade ML model versus Single Multi-class Classification model?
I am developing an ML model for classification using tabular data. It has 5 classes right now and new classes are expected to be continuously added. (Already have a new one leading to an imbalanced ...
- 87
0
votes
0
answers
22
views
What is the best way to create input data samples using in XGBoost for predicting number of next days that customer will come back to store
I'm building the tree-based model like a XGBoost to solve the problem about customer purchase cycle.
And I think, I will build 2 models which one is predicting the customer will come back to store in ...
- 43
0
votes
1
answer
17
views
Additional business rules in ensemble methods (RF, Boosted Trees)
How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees?
These additional business rules can be ...
- 165
2
votes
0
answers
661
views
Binary classification with imbalanced dataset, about lightgbm output probability distribution
I trained a binary classifier for an imbalanced dataset. I did two experiments:
lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample
After training the lgbm model, I ...
- 21
2
votes
2
answers
43
views
confused on "real score" vs "decision value" in classification trees
I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is:
A ...
- 121
0
votes
0
answers
16
views
Model variance increases during training
I trained a regression model with lightgbm and the learning curve doesn't look good:
The model variance increases during training, which shows a kind of overfitting. Now, I tried many ways to fix ...
- 121
0
votes
0
answers
57
views
LightGBM boosting and bagging parameters
When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg.
...
1
vote
0
answers
52
views
Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?
I need to implement a custom loss function.
The function is relatively simple:
$$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$
With $O$ being ...
- 179
0
votes
1
answer
27
views
Scikit-learn estimator not changing predictions when random_state variable changes
I am trying to compute prediction intervals for a classifier I trained in scikit-learn. Even after setting a new random_state parameter in my pipeline, this does ...
- 1
0
votes
0
answers
17
views
LGBM permutation invariance
I'm trying to train a binary classifier with LGBM over pairs of points in $\mathbb{R}^d$ (i.e. a classifier that takes a pair in $\mathbb{R}^{2d}$ and returns a class probability for the pair). I ...
0
votes
1
answer
270
views
Why does Light GBM model produce different results while testing?
Using the Light GBM regressor, I have trained my data and, using Grid Search, I got the best parameters, but while testing with the best parameters I am getting different results each time, which ...
1
vote
1
answer
320
views
Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?
I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
- 35
0
votes
1
answer
28
views
Random Forest but keep only leaves with impurities below a threshold
Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine?
In other words, if I set min ...
5
votes
2
answers
195
views
What is the best way (cheapest / fastest option) to train an model on massive dataset (400GB+, 100m rows x 200 columns)?
I have a 400GB data set that I want to train a model on. What is the cheapest method to train this model? The options I can think of so far are:
AWS instance with massive RAM and train CPU (slow, but ...
- 181
1
vote
0
answers
46
views
cost sensitive loss function in lightbm with individual cost
i am looking for a cost sensitive function that will have weights according to individual row feature (like amount)
this way i can penalize more FN which has large amount vs. low dollar amount.
took ...
- 11
0
votes
1
answer
133
views
Model Dump Parser (like XGBFI) for LightGBM and CatBoost
Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
- 1,517
0
votes
1
answer
55
views
Why does Catboost outperform other boosting algorithms?
I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
- 13
0
votes
1
answer
127
views
Hyperparameter tuning with Bayesian-Optimization
I'm using LightGBM for the regression problem and here is my code.
...
- 3
1
vote
1
answer
79
views
Does Gradient Boosting perform n-ary splits where n > 2?
I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
- 221
1
vote
1
answer
193
views
Negative R2_score Bad predictions for my Sales prediction problem using LightGBM
My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows:
<...
- 11
1
vote
0
answers
105
views
Feature Selection before modeling with Boosting Trees
I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
- 11
1
vote
1
answer
202
views
How to specify scale_pos_weight value at runtime in Hyperopt?
I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown ...
- 11
0
votes
1
answer
270
views
training gradient boosting algorithm in python testing in Golang
What are the best strategy to train and save a gradient boosting algorithm, e.g. LightGBM or XGboost or Catboost in Python but load the model in GoLang and make prediction with Golang ?
- 331
1
vote
2
answers
2k
views
handling missing values for LightGBM model
I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc..,
I have seen some people using ...
- 55
0
votes
1
answer
378
views
Correct theoretical regularized objective function for XGB/LGBM (regression task)
I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
- 21
2
votes
1
answer
77
views
How can i tell if my model is overfitting from the distribution of predicted probabilities?
all,
i am training light gradient boosting and have used all of the necessary parameters to help in over fitting.i plot the predicted probabilities (i..e probabililty has cancer) distribution from the ...
- 466
2
votes
1
answer
2k
views
Lightgbm confidence interval
I want to compute a confidence interval for each sample for a lightgbm model I've trained.
If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
- 5,654
1
vote
0
answers
93
views
Does zero_as_missing parameter affects categorical features? LightGBM
Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names.
LightGBM documentation says ...
12
votes
2
answers
4k
views
What is the proper way to use early stopping with cross-validation?
I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
- 121