Questions tagged [lightgbm]
The lightgbm tag has no usage guidance.
105
questions
2
votes
0
answers
62
views
How to Adjust LightGBM Parameters Like `min_child_samples` When Increasing Training Data?
I'm training a LightGBM model and currently facing a dilemma regarding parameter tuning when increasing the amount of training data.
Initially, I split my dataset into 90% for training and 10% for ...
- 121
0
votes
1
answer
35
views
What is the effect on a LightGBM ranker model of having many groups where all observations have the same target value?
I'm working on a recommender system where users are recomended 5 items at a time.
We build a dataset by saving the 5 items along with feature values and a "relevant" target value (1 if the ...
- 1
0
votes
0
answers
79
views
In my case XGBoost is faster than LightGBM. How do I achieve better speed?
I hope that this place is the right one to ask the following question:
I'm training a machine learning model.
The train size is 85000x55. 49x ...
- 101
0
votes
0
answers
27
views
LightGBM binary classifier: how can I know which values of categorical feature offer the most discrimination?
I am familiar with using methods such as lgb.plot_split_value_histogram to investigate the most used splitting points along numerical features. Is there an ...
0
votes
1
answer
60
views
SHAP values are explaining the wrong output value
I was checking the local accuracy property of the SHAP values. It states that for a data point $(X,y)$, the SHAP values $(s_1,s_2,s_3,...)$ of features $(x_1,x_2,x_3,...)$ sum up to the difference of ...
- 11
2
votes
0
answers
92
views
Tuning the learning rate parameter for GBDT models
I've always been taught that decreasing the learning rate parameter in gbdt models such as XGBoost, LightGBM and Catboost will improve the out-of-sample performance, assuming the number of iterations ...
- 21
0
votes
0
answers
35
views
Gradient boosting algorithm implemented in LightGBM
I'm currently reading the documentation of LightGBM and I'm wondering which gradient boosting algorithm is exactly implemented there if choose boosting parameter as "gbdt" or "dart?&...
- 1
1
vote
0
answers
52
views
How do I know the appropriate number of iterations when using Miceforest for imputation?
I want to know how to avoid overfitting without having to increase the number of iterations excessively in Python with the Miceforest library. I know you can make a correlation map of data sets but I ...
0
votes
0
answers
38
views
LightGBM Regressor miscalibratred/underestimating on high fitted values and overestimating on low fitted values
I'm training a pretty standard LightGBM regressor and noticing a strange pattern with the residuals (see images below--I'm bunching the predicted values and taking the observed average for the group). ...
- 101
0
votes
0
answers
82
views
R [Warning] No further splits with positive gain, best gain: -inf in lightgbm training
I read through some answers, it seems many people face this message before.
The answer seems there is no further need to split the tree, so you need to adjust the super parameter to make new splitting....
- 393
0
votes
1
answer
74
views
In lightgbm,what is a uniform drop?
I read through document of lightgbm,it just tells this parameter, but didn't give much explanation for it.
The explanation:
...
- 393
0
votes
0
answers
271
views
Missing values handling in LightGBM
I'm a bit confused about the handling of missing data by LightGBM. I'm using the R package but my question should not be language-specific.
In a regression setting with no categorical feature, I have ...
- 101
0
votes
0
answers
12
views
Histogram creation in lightgbm in the train API and the scikit-learn API. Is it always benefitial to use the train API?
In the LightGBM for python we have a scikit-learn API in which (either for regression or for classification) there is fit method whose documentation is
fit(X, y, sample_weight=None, init_score=None, ...
- 883
0
votes
1
answer
359
views
How to assign sample weight for regression problem
I'm trying to model a forecasting problem where I'm trying to forecast for the following month. I am using LightGBM Regressor class for it and it giving me a decent ...
1
vote
0
answers
114
views
0
votes
1
answer
241
views
Is there a way to focus mainly on high precision when fitting a tree model?
I have a dataset with 95% false and 5% true labels, some 200000 samples overall, I'm fitting a LightGBM model. I mainly need to focus on high precision and have low number of false positives, I don't ...
- 1
0
votes
1
answer
389
views
randomness in lightgbm model training
What are the parameters that add randomness to the training of a lightgbm model? (for a large dataset) I have tried setting all parameters as default and letting bin_construct_sample_cnt be greater ...
- 1
0
votes
0
answers
1k
views
lightgbm issue ValueError: Input numpy.ndarray or list must be 2 dimensional
Problem
Build prediction accuracy model using lightgbm on New York Taxi Duration dataset. Problem is "ValueError: Input numpy.ndarray or list must be 2 dimensional" with lightgbm.predict() ...
1
vote
1
answer
465
views
Why is monotonic constraint disabled when using MAE as an objective to LGBM?
I tried to use monotonic constraints in LGBM, but if I use mean absolute error as an objective, it gives a warning that monotonic constraints cannot be done in l1.
What is the reason? Thanks!
- 25
0
votes
1
answer
81
views
Combining results from classifiers trained on different test/train splits results in higher accuracy
I have developed a classifier model using LightGBM. The accuracy of the model varies significantly because of the test_train_split state(between 83% and 91%). This ...
0
votes
1
answer
116
views
Why the order of the fearures affects synapse LightGBM predictions?
I am using LighGBM Classifier and Regressor and it seems that the order of the features I am adding, affect the predictions of the model. Everytime I change the order, another result comes up and with ...
4
votes
1
answer
746
views
Optuna Median Pruner n_warmup_steps
For Gradient Boosting Models such as XGBOOST and LGBM does n_warmup_steps in optuna.pruners.MedianPruner refer to the minimum number of folds evaluated before pruning is triggered?
I.e. if number of ...
- 140
1
vote
1
answer
776
views
Grid-search for a multi-output regression task using Scikit-learn's API
I'm trying to make a model for a multi-output regression task where $y=(y_1, y_2,..., y_n)$ is a vector rather than a single scalar. I am using Scikit-learn's ...
- 125
0
votes
0
answers
53
views
Multiclass classification (gradient boosted trees) predictions distribution using softmax()
Let's consider a multiclass problem where the target is composed by 20% class 'A', 50% of class 'B' and 30% of class 'C'.
The model is trained and then the class predictions are obtained via the ...
- 133
0
votes
1
answer
267
views
Impact of many zeros in LightGBM Regressor training set [duplicate]
I have a LightGBM Regressor model with 15 features. 5 of these features have 98.7% NA for the training set. All five of the features are NA for each row. I impute the missing values with zero before I ...
- 1
1
vote
0
answers
241
views
Sklearn Lightgbm 3.3.2 API
I have a model object from a model and am trying to pass in the dataframe for scoring, it is kicking me an error:
...
- 111
2
votes
1
answer
992
views
Online Learning/Continual Learning for tree-based Algorithms
Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
- 368
1
vote
2
answers
1k
views
How to improve Regression RMSE with LightGBM
I have the following dataset:
https://raw.githubusercontent.com/Joffreybvn/real-estate-data-analysis/master/data/clean/belgium_real_estate.csv
I want to predict the price column, based on the other ...
- 133
1
vote
0
answers
130
views
How to use GBDT with EFB or GOSS with EFB in LightGBM? or is it implemented by default?
The paper for Lightgbm talks about goss and efb, I want to know how to use these together. I know of the hyper-parameter 'boosting' can be used to set boosting as gbdt, or goss, or dart. But how to ...
- 77
1
vote
1
answer
835
views
How can I implement lambda-mart with lightgbm?
I have a learning to rank task at hand and I want to use the lightgbm implementation of LambdaMART. I'm also following this notebook.
...
- 696
1
vote
0
answers
557
views
How does exactly eval_set and RandomizedSearchCV work for LightGBM?
How does RandomizedSearchCV form the validation sets, while I also defined an evaluation set for LGBM? Is it formed from the train set I gave or how does the evaluation set comes into the validation?
...
- 25
0
votes
0
answers
54
views
Constructing pandas DF with model trained categorical variable
I've trained a lightGBM model on a dataset X where X has a categorical (in the pandas sense) variable. This model trains fine and when I predict using it all looks good - I can even change the value ...
0
votes
1
answer
263
views
SOS: Working LightGBM model script to find best model
I have been trying to get a working LightGBM model which I can train on my data, select the best performing model with highest f1 score and then use it obtain the f1 score on the testing data. However,...
- 1
1
vote
1
answer
247
views
Model Performance on external validation Set really low?
I am using the LGBM model for binary classification. My train and test accuracies are 87% & 82% respectively with cross-validation of 89%. ROC-AUC score of 81%. But when evaluating model ...
- 77
0
votes
3
answers
1k
views
Is my model overfitting ? Training Acc :93 % test accuracy 82%
I am using LGBM model for binary classification. After hyper-parameter tuning I get
Training accuracy 0.9340
Test accuracy 0.8213
can I say my model is overfitting?...
- 77
0
votes
0
answers
302
views
LightGBM predict_proba in thousandths place
Can someone explain to me how my lightgbm classification model's predict_proba() is in thousandths place for the positive class:
...
- 111
2
votes
1
answer
305
views
Random LightGBM Forest
I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
- 510
1
vote
1
answer
113
views
Proof of GOSS algorithm in lightGBM paper
In the LightGBM paper the authors make use of a newly developed sampling method GOSS to reduce the number of data instances needed for finding the best split of a ...
- 121
0
votes
1
answer
237
views
Incorporating data over time into lightgbm
So I'm in the situation where I know what it is I'm trying to find, but not the terminology for it and I think that's why a lot of my google searches are directing me in the wrong direction, so ...
- 161
1
vote
1
answer
4k
views
LightGBM eval_set - what to do when I fit the final model (there's no test data left)
I'm using LightGBM's eval_set feature when fitting my model. This enables early stopping on the number of estimators used.
...
- 245
1
vote
1
answer
264
views
Understanding feature_parallel distributed learning algorithm in LightGBMClassifier
I want to understand feature_parallel algorithm in LightGBMClassifier.
It describes how it is done traditionally and how LightGBM...
- 883
3
votes
3
answers
191
views
Example for Boosting
Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
- 193
1
vote
0
answers
46
views
Fractional Differencing/Differentiation for Non-Time based Model; Look-ahead bias?
I have time-series data, but instead of using a time-based model like RNN, I've decided to approach my classification problem using an lgbm classifier. To do so, I have modified the data, such that ...
- 11
2
votes
2
answers
737
views
Optimizing MAE degrades MAE metrics
I have run a lighgbm regression model by optimizing on RMSE and measuring the performance on RMSE:
...
- 121
0
votes
1
answer
3k
views
LightGBM regressor score function?
I'm trying to find what is the score function for the LightGBM regressor. In their documentation page I could not find any information regarding the function used to calculate the score attribute...
0
votes
2
answers
157
views
Churn prediction model doesn't predict good on real data
I am working currently on churn prediction problem.
As an input I use data from date warehouse for a period 082016 - 032021(one row per month for each customer).
Based on this data I have created a ...
- 113
0
votes
1
answer
23
views
Additional business rules in ensemble methods (RF, Boosted Trees)
How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees?
These additional business rules can be ...
- 165
2
votes
0
answers
2k
views
Binary classification with imbalanced dataset, about lightgbm output probability distribution
I trained a binary classifier for an imbalanced dataset. I did two experiments:
lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample
After training the lgbm model, I ...
- 21
2
votes
2
answers
278
views
confused on "real score" vs "decision value" in classification trees
I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is:
A ...
- 153
1
vote
0
answers
79
views
Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?
I need to implement a custom loss function.
The function is relatively simple:
$$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$
With $O$ being ...
- 199