Questions tagged [lightgbm]

The tag has no usage guidance.

30 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
1answer
988 views

Math Behind GOSS (Gradient-Based One Side Sampling)?

As per my understanding through books & Google Search, GOSS (Gradient-Based One Side Sampling) is a novel sampling method that downsamples the instances on the basis of gradients. As we know ...
2
votes
0answers
164 views

Comparing feature importance in LightGBM + Scikit

I have a model trained using LightGBM (LGBMRegressor), in Python, with scikit-learn. On a weekly basis the model in re-trained, and an updated set of chosen features and associated ...
1
vote
0answers
33 views

Which Neural Network or Gradient Boosting framework is the simplest for Custom Loss Functions?

I need to implement a custom loss function. The function is relatively simple: $$-\sum \limits_{i=1}^m [O_{1,i} \cdot y_i-1] \ \cdot \ \operatorname{ReLu}(O_{1,i} \cdot \hat{y_i} - 1)$$ With $O$ being ...
1
vote
1answer
25 views

while Light GBM model produces different results while Testing

Using Light GBM regressor I'm trained my data & using Grid Search I got best parameters, but while testing with best parameters I'm getting different results for each time that means model ...
1
vote
1answer
25 views

Why is HistGradientBoostingRegressor in sklearn so fast and low on memory?

I trained multiple models for my problem and most ensemble algorithms resulted in lengthy fit and train time and huge model size on disk (approx 10GB for RandomForest) but when I tried ...
1
vote
0answers
25 views

cost sensitive loss function in lightbm with individual cost

i am looking for a cost sensitive function that will have weights according to individual row feature (like amount) this way i can penalize more FN which has large amount vs. low dollar amount. took ...
1
vote
1answer
113 views

Negative R2_score Bad predictions for my Sales prediction problem using LightGBM

My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows: <...
1
vote
0answers
52 views

Feature Selection before modeling with Boosting Trees

I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
1
vote
1answer
79 views

How to specify scale_pos_weight value at runtime in Hyperopt?

I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown ...
1
vote
0answers
49 views

Does zero_as_missing parameter affects categorical features? LightGBM

Dealing with categorical features while training LightGBM model implies encoding them as integers and providing categorical_feature parameter with their indices or names. LightGBM documentation says ...
1
vote
0answers
182 views

Gradient boosting Regression with zero-inflated outcome

I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
1
vote
0answers
572 views

Read back a saved LGBMClassifier model

I trained a LGBMClassifier model and saved in a file in this way: ...
1
vote
0answers
350 views

Use LightGBM or FFM - imbalanced dataset

I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
1
vote
0answers
447 views

How to save a lightGBM model that updates predictions after each fold?

Hi When I use gradient boosting on Kaggle and large data sets, I run a code like this: ...
1
vote
0answers
412 views

How does L1 Loss work in lightGBM

From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance. My question is that, when the objective is L1 loss/...
0
votes
0answers
19 views

LightGBM boosting and bagging parameters

When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg. ...
0
votes
1answer
20 views

Scikit-learn estimator not changing predictions when random_state variable changes

I am trying to compute prediction intervals for a classifier I trained in scikit-learn. Even after setting a new random_state parameter in my pipeline, this does ...
0
votes
0answers
12 views

LGBM permutation invariance

I'm trying to train a binary classifier with LGBM over pairs of points in $\mathbb{R}^d$ (i.e. a classifier that takes a pair in $\mathbb{R}^{2d}$ and returns a class probability for the pair). I ...
0
votes
0answers
16 views

Calibration of a few binary classifiers is not perfect - why?

I am working on a binary classifier using LightGBM. I try to see the results of the classifiers when changing the costs of false positives and false negatives, still working on the same training and ...
0
votes
1answer
20 views

Random Forest but keep only leaves with impurities below a threshold

Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine? In other words, if I set min ...
0
votes
0answers
22 views

LightGBM model improvement when the focus is on probability prediction

I am building a binary classifier using LightGBM. The goal is not to predict the outcome as such, but rather to predict the probability of the target even. To be more specific, it's more about ranking ...
0
votes
0answers
37 views

LightGBM get model decision(rules)

I need to interpret the model decision for binary classification. Here is my model: ...
0
votes
0answers
13 views

Tree Based Classification (XGBoost, LightGBM, etc) - Features from embeddings for sparse features?

I'm wondering if there is a possibility from using embeddings as inputs for tree based classification models? For example we have a field called type of food, and ...
0
votes
1answer
58 views

Model Dump Parser (like XGBFI) for LightGBM and CatBoost

Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
0
votes
0answers
24 views

Why does Catboost outperform other boosting algorithms?

I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
0
votes
0answers
13 views

Gbm prediction distribution different to training data

I’m doing a regression on a dataset using lightgbm. For the training data the response variable has a non normal distribution which is multimodal. However, the predictions out-of-fold have are ...
0
votes
0answers
130 views

how to choose importance_type in lgbm.plot_importance

I have made a binary-classifier using lgbm.The classifier is made on unbalanced dataset. I wanted to see the importance features of the model. There are two types of selecting importance_type - ...
0
votes
0answers
241 views

Understanding interplay between eval_metric, metric and first_metric_only parameters in LGBMClassifier

In python API of LGBMClassifier, the constructor takes parameters metric and first_metric_only. Their descriptions are as ...
0
votes
1answer
231 views

Correct theoretical regularized objective function for XGB/LGBM (regression task)

I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
0
votes
1answer
149 views

Sliding window approach using SVR & LightGBM

I'm working on a multivariate time series forecast using a couple of ML algorithms (Neural Networks, Support Vector Machines & Gradient boosting algorithms). I need to measure the performance of ...