Questions tagged [lightgbm]
The lightgbm tag has no usage guidance.
70
questions
0
votes
0answers
21 views
what is the meaning of split maximum return leaf, non split maximum return leaf in lightgbm
I am currently working on lightgbm, where i read this sentence i am unable to understand what this means. can anyone help me that what it means.
thank you.
https://www.hindawi.com/journals/mpe/2020/...
0
votes
2answers
25 views
Churn prediction model doesn't predict good on real data
I am working currently on churn prediction problem.
As an input I use data from date warehouse for a period 082016 - 032021(one row per month for each customer).
Based on this data I have created a ...
0
votes
0answers
7 views
For a Multi-class Classification Problem, What are the Pros and Cons of using a Cascade ML model versus Single Multi-class Classification model?
I am developing an ML model for classification using tabular data. It has 5 classes right now and new classes are expected to be continuously added. (Already have a new one leading to an imbalanced ...
0
votes
0answers
12 views
What is the best way to create input data samples using in XGBoost for predicting number of next days that customer will come back to store
I'm building the tree-based model like a XGBoost to solve the problem about customer purchase cycle.
And I think, I will build 2 models which one is predicting the customer will come back to store in ...
0
votes
1answer
11 views
Additional business rules in ensemble methods (RF, Boosted Trees)
How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees?
These additional business rules can be ...
2
votes
0answers
86 views
Binary classification with imbalanced dataset, about lightgbm output probability distribution
I trained a binary classifier for an imbalanced dataset. I did two experiments:
lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample
After training the lgbm model, I ...
2
votes
2answers
31 views
confused on "real score" vs "decision value" in classification trees
I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is:
A ...
0
votes
0answers
12 views
Model variance increases during training
I trained a regression model with lightgbm and the learning curve doesn't look good:
The model variance increases during training, which shows a kind of overfitting. Now, I tried many ways to fix ...
0
votes
0answers
25 views
LightGBM boosting and bagging parameters
When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg.
...
1
vote
0answers
38 views
Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?
I need to implement a custom loss function.
The function is relatively simple:
$$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$
With $O$ being ...
0
votes
1answer
26 views
Scikit-learn estimator not changing predictions when random_state variable changes
I am trying to compute prediction intervals for a classifier I trained in scikit-learn. Even after setting a new random_state parameter in my pipeline, this does ...
0
votes
0answers
13 views
LGBM permutation invariance
I'm trying to train a binary classifier with LGBM over pairs of points in $\mathbb{R}^d$ (i.e. a classifier that takes a pair in $\mathbb{R}^{2d}$ and returns a class probability for the pair). I ...
1
vote
1answer
55 views
Why does Light GBM model produce different results while testing?
Using the Light GBM regressor, I have trained my data and, using Grid Search, I got the best parameters, but while testing with the best parameters I am getting different results each time, which ...
1
vote
1answer
44 views
Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?
I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
0
votes
0answers
17 views
Calibration of a few binary classifiers is not perfect - why?
I am working on a binary classifier using LightGBM. I try to see the results of the classifiers when changing the costs of false positives and false negatives, still working on the same training and ...
0
votes
1answer
21 views
Random Forest but keep only leaves with impurities below a threshold
Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine?
In other words, if I set min ...
0
votes
0answers
51 views
LightGBM model improvement when the focus is on probability prediction
I am building a binary classifier using LightGBM. The goal is not to predict the outcome as such, but rather to predict the probability of the target even. To be more specific, it's more about ranking ...
0
votes
0answers
49 views
LightGBM get model decision(rules)
I need to interpret the model decision for binary classification. Here is my model:
...
6
votes
1answer
123 views
What is the best way (cheapest / fastest option) to train an model on massive dataset (400GB+, 100m rows x 200 columns)?
I have a 400GB data set that I want to train a model on. What is the cheapest method to train this model? The options I can think of so far are:
AWS instance with massive RAM and train CPU (slow, but ...
0
votes
0answers
30 views
Tree Based Classification (XGBoost, LightGBM, etc) - Features from embeddings for sparse features?
I'm wondering if there is a possibility from using embeddings as inputs for tree based classification models?
For example we have a field called type of food, and ...
1
vote
0answers
31 views
cost sensitive loss function in lightbm with individual cost
i am looking for a cost sensitive function that will have weights according to individual row feature (like amount)
this way i can penalize more FN which has large amount vs. low dollar amount.
took ...
0
votes
1answer
83 views
Model Dump Parser (like XGBFI) for LightGBM and CatBoost
Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
0
votes
1answer
40 views
Why does Catboost outperform other boosting algorithms?
I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
0
votes
1answer
77 views
Hyperparameter tuning with Bayesian-Optimization
I'm using LightGBM for the regression problem and here is my code.
...
0
votes
0answers
15 views
Gbm prediction distribution different to training data
I’m doing a regression on a dataset using lightgbm. For the training data the response variable has a non normal distribution which is multimodal. However, the predictions out-of-fold have are ...
1
vote
1answer
35 views
Does Gradient Boosting perform n-ary splits where n > 2?
I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
1
vote
1answer
141 views
Negative R2_score Bad predictions for my Sales prediction problem using LightGBM
My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows:
<...
0
votes
0answers
159 views
how to choose importance_type in lgbm.plot_importance
I have made a binary-classifier using lgbm.The classifier is made on unbalanced dataset.
I wanted to see the importance features of the model. There are two types of selecting importance_type -
...
1
vote
0answers
64 views
Feature Selection before modeling with Boosting Trees
I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
1
vote
1answer
104 views
How to specify scale_pos_weight value at runtime in Hyperopt?
I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown ...
0
votes
1answer
182 views
training gradient boosting algorithm in python testing in Golang
What are the best strategy to train and save a gradient boosting algorithm, e.g. LightGBM or XGboost or Catboost in Python but load the model in GoLang and make prediction with Golang ?
1
vote
2answers
1k views
handling missing values for LightGBM model
I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc..,
I have seen some people using ...
0
votes
1answer
277 views
Correct theoretical regularized objective function for XGB/LGBM (regression task)
I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
2
votes
1answer
62 views
How can i tell if my model is overfitting from the distribution of predicted probabilities?
all,
i am training light gradient boosting and have used all of the necessary parameters to help in over fitting.i plot the predicted probabilities (i..e probabililty has cancer) distribution from the ...
1
vote
1answer
2k views
Lightgbm confidence interval
I want to compute a confidence interval for each sample for a lightgbm model I've trained.
If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard ...
1
vote
0answers
57 views
Does zero_as_missing parameter affects categorical features? LightGBM
Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names.
LightGBM documentation says ...
10
votes
2answers
2k views
What is the proper way to use early stopping with cross-validation?
I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
0
votes
1answer
33 views
Should we do Feature selection in parallel with feature engineering?
I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i ...
3
votes
1answer
2k views
How is the "base value" of SHAP values calculated?
I'm trying to understand how the base value is calculated. So I used an example from SHAP's github notebook, Census income classification with LightGBM.
Right after I trained the lightgbm model, I ...
2
votes
1answer
1k views
How to use r2-score as a loss function in LightGBM?
I am trying to implement a custom loss function in LightGBM for a regression problem. The intrinsic metrics do not help me much, because they penalise for outliers... Is there any way to use ...
1
vote
0answers
198 views
Gradient boosting Regression with zero-inflated outcome
I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
4
votes
1answer
2k views
Small number of estimators in gradient boosting
I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
1
vote
1answer
71 views
0
votes
1answer
966 views
random forest and log_loss metric?
Light gbm has the metric with log_loss for binary or multi classification.
is Random Forest also has the loss function with log_loss?
1
vote
1answer
142 views
Number of leaves for lightgbm is smaller than categories in one feature
I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
3
votes
1answer
865 views
What does the repeated message "No further splits with positive gain, best gain: -inf" mean?
I am training a LightGBM classifier on a binary classification problem.
From time to time I get the following message repeatedly:
...
5
votes
1answer
4k views
How do GBM algorithms handle missing data?
How do algorithms GBM algorithms, such as XGBoost or LightGBM handle NaN values?
I know that they learn how to replace NaN values with other values but my question is: How do they do it exactly?
4
votes
2answers
2k views
Correct interpretation of summary_plot shap graph
While through the various resources online to understand the shap plots, I ended up slightly confused. Find below my interpretation of the overall plot given in examples -
Shap value 0 for a feature ...
2
votes
1answer
181 views
Extremely high gain with LightGBM
I am working on a binary classification problem. The target variable is not linearly separable, so I've decided to use LightGBM with default parameters (I only play with n_estimators on range from 10 -...
6
votes
1answer
1k views
Math Behind GOSS (Gradient-Based One Side Sampling)?
As per my understanding through books & Google Search,
GOSS (Gradient-Based One Side Sampling) is a novel sampling method that downsamples the instances on the basis of gradients. As we
know ...
