Questions tagged [lightgbm]
The lightgbm tag has no usage guidance.
101
questions
0
votes
0
answers
9
views
lightgbm issue ValueError: Input numpy.ndarray or list must be 2 dimensional
Problem
Build prediction accuracy model using lightgbm on New York Taxi Duration dataset. Problem is "ValueError: Input numpy.ndarray or list must be 2 dimensional" with lightgbm.predict() ...
0
votes
0
answers
6
views
Approaching multiple records for one observation; radiomics of 2D slices of a 3D object
Background
I am trying to create a model that can predict Type 2 diabetes in a patient based on MRI scans of their thigh muscle. Previous literature has shown that fat deposition in the muscle of ...
0
votes
0
answers
24
views
Lightgbm model tuning produces unexpectedly different hyperparameters for similar datasets
I am trying to tune a lightgbm model for each half of a dataset that I have split by a particular feature (stock ticker in this case). Both halves have the same number of features, somewhat similar ...
1
vote
1
answer
43
views
Why is monotonic constraint disabled when using MAE as an objective to LGBM?
I tried to use monotonic constraints in LGBM, but if I use mean absolute error as an objective, it gives a warning that monotonic constraints cannot be done in l1.
What is the reason? Thanks!
- 25
0
votes
1
answer
22
views
Combining results from classifiers trained on different test/train splits results in higher accuracy
I have developed a classifier model using LightGBM. The accuracy of the model varies significantly because of the test_train_split state(between 83% and 91%). This ...
0
votes
0
answers
14
views
Why the order of the fearures affects synapse LightGBM predictions?
I am using LighGBM Classifier and Regressor and it seems that the order of the features I am adding, affect the predictions of the model. Everytime I change the order, another result comes up and with ...
0
votes
0
answers
28
views
Forecast n-steps into the future with LightGBM
I have learnt to create the LightGBM models with multiple features and target, but in ALL the examples I have seen thus far they only show how to do a "predict" on the x_test_df. What I am ...
- 101
0
votes
0
answers
28
views
Understanding loss function/ implementing my own
I am currently working on an ETA prediction LightGBM model (regression tree) for which I want the negative residuals to be penalized higher than the positive. I understand that a custom loss function ...
0
votes
0
answers
42
views
When using custom metrics, num_threads setting is not working in lightgbm?
When I train a lightgbm model with my own custom metrics, I find that model is trained in single thread, though I set "num_threads": 16.
...
- 21
3
votes
1
answer
132
views
Optuna Median Pruner n_warmup_steps
For Gradient Boosting Models such as XGBOOST and LGBM does n_warmup_steps in optuna.pruners.MedianPruner refer to the minimum number of folds evaluated before pruning is triggered?
I.e. if number of ...
- 131
0
votes
0
answers
23
views
Worse performance on positive class - probability prediction with lightgbm
I would like to predict probabilities in a binary class setting. I want to use the probabilities directly to make decisions, rather than using the exact class label. E.g. I want to vary some features ...
- 1
0
votes
1
answer
199
views
Grid-search for a multi-output regression task using Scikit-learn's API
I'm trying to make a model for a multi-output regression task where $y=(y_1, y_2,..., y_n)$ is a vector rather than a single scalar. I am using Scikit-learn's ...
- 115
0
votes
0
answers
40
views
Multiclass classification (gradient boosted trees) predictions distribution using softmax()
Let's consider a multiclass problem where the target is composed by 20% class 'A', 50% of class 'B' and 30% of class 'C'.
The model is trained and then the class predictions are obtained via the ...
- 133
0
votes
1
answer
45
views
Impact of many zeros in LightGBM Regressor training set
I have a LightGBM Regressor model with 15 features. 5 of these features have 98.7% NA for the training set. All five of the features are NA for each row. I impute the missing values with zero before I ...
- 1
1
vote
0
answers
56
views
Sklearn Lightgbm 3.3.2 API
I have a model object from a model and am trying to pass in the dataframe for scoring, it is kicking me an error:
...
- 111
1
vote
1
answer
231
views
Online Learning/Continual Learning for tree-based Algorithms
Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
- 318
1
vote
2
answers
428
views
How to improve Regression RMSE with LightGBM
I have the following dataset:
https://raw.githubusercontent.com/Joffreybvn/real-estate-data-analysis/master/data/clean/belgium_real_estate.csv
I want to predict the price column, based on the other ...
- 133
0
votes
0
answers
44
views
LightGBM Price Prediction always INF but RMSE 0.19
I have a real state dataset from Belgium:
https://raw.githubusercontent.com/Joffreybvn/real-estate-data-analysis/master/data/clean/belgium_real_estate.csv
And I want to use LightGBM for price ...
- 133
1
vote
0
answers
73
views
How to use GBDT with EFB or GOSS with EFB in LightGBM? or is it implemented by default?
The paper for Lightgbm talks about goss and efb, I want to know how to use these together. I know of the hyper-parameter 'boosting' can be used to set boosting as gbdt, or goss, or dart. But how to ...
- 47
0
votes
0
answers
41
views
Walk-forward validation nrounds LightGBM
Dear Data Science Gurus,
I am facing the following phenomenon:
1-Scenario:
Predict demand for n-month in the future for hundreds of products using lightgbm. I have 2 years of history
2- Approach:
Walk-...
0
votes
0
answers
23
views
Time series forecasting model predicts increasing number for target variable when the actual values are zeroes
I am working on time series forecasting model, and I am using light gbm. The project goal is to predict the number of sales across different levels (very similar to the M5 competition). For instance, ...
- 113
0
votes
0
answers
111
views
How to calculate the feature importance of variables synthesized with EFB in LightGBM?
I've been studying LightGBM recently.
I have a question. Feature Importance can be extracted when running the LightGBM library in Python.
LightGBM has an Exclusive feature bundling feature that allows ...
- 1
1
vote
1
answer
329
views
How can I implement lambda-mart with lightgbm?
I have a learning to rank task at hand and I want to use the lightgbm implementation of LambdaMART. I'm also following this notebook.
...
- 674
1
vote
0
answers
216
views
How does exactly eval_set and RandomizedSearchCV work for LightGBM?
How does RandomizedSearchCV form the validation sets, while I also defined an evaluation set for LGBM? Is it formed from the train set I gave or how does the evaluation set comes into the validation?
...
- 25
0
votes
0
answers
30
views
Constructing pandas DF with model trained categorical variable
I've trained a lightGBM model on a dataset X where X has a categorical (in the pandas sense) variable. This model trains fine and when I predict using it all looks good - I can even change the value ...
0
votes
1
answer
80
views
SOS: Working LightGBM model script to find best model
I have been trying to get a working LightGBM model which I can train on my data, select the best performing model with highest f1 score and then use it obtain the f1 score on the testing data. However,...
- 1
0
votes
0
answers
33
views
Does lightGBM handle multicollinearity? [duplicate]
I have a dataset after feature selection of around 6500 features and 10,000 data rows. I am using LightGBM model. I want to know if I should check the feature set for multicollinearity. If two or more ...
- 47
0
votes
0
answers
131
views
splitting point in LightGBM?
I am not able to understand how the first root node is selected in LightGBM and how the splitting at nodes happens further. I read blogs and related documents and I understand that in this histogram-...
- 47
1
vote
1
answer
32
views
Model Performance on external validation Set really low?
I am using the LGBM model for binary classification. My train and test accuracies are 87% & 82% respectively with cross-validation of 89%. ROC-AUC score of 81%. But when evaluating model ...
- 47
0
votes
0
answers
102
views
LGBM model predicting only single class on unseen data!
I have built a LightGBM based machine learning model on data of molecules of two classes. The distribution is as follows. Class 0 has 5933 data points and class 1 has 4696. The train test accuracy I ...
- 47
0
votes
3
answers
207
views
Is my model overfitting ? Training Acc :93 % test accuracy 82%
I am using LGBM model for binary classification. After hyper-parameter tuning I get
Training accuracy 0.9340
Test accuracy 0.8213
can I say my model is overfitting?...
- 47
0
votes
0
answers
201
views
LightGBM predict_proba in thousandths place
Can someone explain to me how my lightgbm classification model's predict_proba() is in thousandths place for the positive class:
...
- 111
1
vote
1
answer
156
views
Random LightGBM Forest
I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
- 440
1
vote
1
answer
69
views
Proof of GOSS algorithm in lightGBM paper
In the LightGBM paper the authors make use of a newly developed sampling method GOSS to reduce the number of data instances needed for finding the best split of a ...
- 121
0
votes
1
answer
62
views
Incorporating data over time into lightgbm
So I'm in the situation where I know what it is I'm trying to find, but not the terminology for it and I think that's why a lot of my google searches are directing me in the wrong direction, so ...
- 151
1
vote
1
answer
2k
views
LightGBM eval_set - what to do when I fit the final model (there's no test data left)
I'm using LightGBM's eval_set feature when fitting my model. This enables early stopping on the number of estimators used.
...
- 245
1
vote
1
answer
118
views
Understanding feature_parallel distributed learning algorithm in LightGBMClassifier
I want to understand feature_parallel algorithm in LightGBMClassifier.
It describes how it is done traditionally and how LightGBM...
- 775
3
votes
3
answers
93
views
Example for Boosting
Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
- 193
1
vote
0
answers
22
views
Fractional Differencing/Differentiation for Non-Time based Model; Look-ahead bias?
I have time-series data, but instead of using a time-based model like RNN, I've decided to approach my classification problem using an lgbm classifier. To do so, I have modified the data, such that ...
- 11
2
votes
1
answer
223
views
Optimizing MAE degrades MAE metrics
I have run a lighgbm regression model by optimizing on RMSE and measuring the performance on RMSE:
...
- 121
0
votes
1
answer
1k
views
LightGBM regressor score function?
I'm trying to find what is the score function for the LightGBM regressor. In their documentation page I could not find any information regarding the function used to calculate the score attribute...
0
votes
2
answers
95
views
Churn prediction model doesn't predict good on real data
I am working currently on churn prediction problem.
As an input I use data from date warehouse for a period 082016 - 032021(one row per month for each customer).
Based on this data I have created a ...
- 113
0
votes
1
answer
19
views
Additional business rules in ensemble methods (RF, Boosted Trees)
How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees?
These additional business rules can be ...
- 165
2
votes
0
answers
1k
views
Binary classification with imbalanced dataset, about lightgbm output probability distribution
I trained a binary classifier for an imbalanced dataset. I did two experiments:
lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample
After training the lgbm model, I ...
- 21
2
votes
2
answers
80
views
confused on "real score" vs "decision value" in classification trees
I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is:
A ...
- 121
1
vote
0
answers
67
views
Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?
I need to implement a custom loss function.
The function is relatively simple:
$$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$
With $O$ being ...
- 189
0
votes
1
answer
32
views
Scikit-learn estimator not changing predictions when random_state variable changes
I am trying to compute prediction intervals for a classifier I trained in scikit-learn. Even after setting a new random_state parameter in my pipeline, this does ...
- 1
0
votes
1
answer
714
views
Why does Light GBM model produce different results while testing?
Using the Light GBM regressor, I have trained my data and, using Grid Search, I got the best parameters, but while testing with the best parameters I am getting different results each time, which ...
1
vote
1
answer
938
views
Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?
I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
- 35
0
votes
1
answer
37
views
Random Forest but keep only leaves with impurities below a threshold
Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine?
In other words, if I set min ...
