Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1 vote
3 answers
58 views

Using Transaction Amount to Guide Learning in an Fraud Detection Machine Learning Model

I am currently using transaction amount as a feature in an XGBoost classification model designed to identify fraudulent transactions. Furthermore, transaction amount is bounded for this problem ...
6 votes
1 answer
231 views

What is a good objective function for allowing close to 0 predictions?

Let's say we want to predict the probability of rain. So just the binary case: rain or no rain. In many cases it makes sense to have this in the [5%, 95%] interval. And for many applications this ...
0 votes
0 answers
18 views

How can I write a custom loss function to punish lower predicted values?

I am trying to write a custom loss function for XGBRegressor that needs to punish predicted values that are under some arbitrary threshold. The code I came up with does not affect the results at all, ...
2 votes
1 answer
1k views

Xgboost Multiclass evaluation Metrics

Im training an Xgb Multiclass problem, but im having doubts about my evaluation metrics, heres my code + output ...
2 votes
0 answers
541 views

What is the difference between RandomForestClassifier and XGBRFClassifier?

What is the difference between RandomForestClassifier and XGBRFClassifier? There is no detailed explanation about what XGBRFClassifier exactly is so I was wondering.
1 vote
1 answer
426 views

Correct theoretical regularized objective function for XGB/LGBM (regression task)

I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
0 votes
1 answer
180 views

XGBOOST with target column has categorical data and features also has categorical data

I have a huge dataset with the categorical columns in features and also my target variable is categorical. All the values are not ordinal so I think it is best to use one hot encoding. But I have one ...
2 votes
2 answers
4k views

Minimum number of samples to train XGBoost without overfitting

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron. Is there a similar rule of thumb for classifiers ...
0 votes
1 answer
25 views

XGBoost training on sample of time series data

I am new to XGBoost and would like to use it on a time series dataset. Here is the scenario I'm faced with: The data set contains N samples of length T, with N>>T. I'd like to train an XGBoost ...
0 votes
2 answers
660 views

Consequences of using XGBoost regressor for small dataset(< 500 rows)

I am using XGBoost regressor to train my model for 322 rows of data and the train and test split is as follows: ((257, 9), (257,), (65, 9), (65,)) I am using the ...
3 votes
0 answers
46 views

Does ROC AUC different between crossval and test set indicate overfitting or other problem?

I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
1 vote
1 answer
50 views

Online Learning/Continual Learning for tree-based Algorithms

Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
2 votes
2 answers
7k views

How to get xgbregressor feature importance by column name?

...
1 vote
1 answer
146 views

scale_pos_weight using XGBoost's Learning API

I see it is possible to add a weight for unbalanced problems in XGBoost's Scikit-Learn API through scale_pos_weight. Does it have an equivalent in the Learning API? If not, is there a reason behind ...
3 votes
1 answer
585 views

Multiclass Classification with Decision Trees: Why do we calculate a score and apply softmax?

I'm trying to figure out why when using decision trees for multi class classification it is common to calculate a score and apply softmax, instead of just taking the averages of the terminal nodes ...
0 votes
0 answers
7 views

Part random sample, part force sample with multiple event=0

Question: Is this approach sound? Goal: Model probability of event = 1 Problem: Time series with multiple event = 0 per id, only one event=1 Approach: Get all records where event = 1 (not random ...
1 vote
1 answer
24 views

scale_pos_weight effect in XGBClassifier

I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier. It says everywhere to set it to Count of negatives / Count of ...
0 votes
4 answers
126 views

Can this problem be solved using deep learning?

I want to predict price of used cars. I have data like this: Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.? I used one hot approach for nominal features and ...
0 votes
1 answer
490 views

How does XGBoost perform in Parallel

So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
1 vote
1 answer
248 views

Are there rules of thumb for xgboosts hyperperameter selection?

There are multiple parameters that need to be specified in the XGBClassifier. Certainly gridsearchcv could give some insight into optimal hyperparameters, but I would imagine there are some rules of ...
2 votes
1 answer
140 views

My own model trained on the full data is better than the best_estimator I get from GridSearchCV with refit=True?

I am using an XGBoost model to classify some data. I have cv splits (train, val) and a separate test set that I never use until the end. I have used GridSearchCV to determine the best parameters and ...
2 votes
1 answer
600 views

Marketing Mix Model (MMM) using XGBoost?

Does anyone know of any literature on Marketing Mix Modelling (MMM) using XGBoost? Is this a viable techniques for MMM modelling? What would the advantages/disadvantages be? I think it would be a ...
1 vote
1 answer
1k views

Using a combination of gradient boosting with LSTM for classification?

I am presently using an LSTM model to classify high dimensional tabular data which is not text/images (dimensions 21392x1970). I also tried XGBoost (Gradient boosting) in Python separately for the ...
1 vote
1 answer
74 views

Tuning parameters for gradient boosting/xgboost

In practice, which parameter do you typically tune first? Do you tune the learning rate (or step size) first? and then tune the total number of iterations? And how do you go about tuning these ...
0 votes
1 answer
12 views

combine xgb feature importance

Is it possible to combine the results from the xgb.boost importance function. For example, due to one hot encoding, I have a feature age=35 and another age=60. Is there a way that I can add these to ...
0 votes
1 answer
1k views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where I am trying to predict if the data returns a 1 or 0. So your classic binary classification. I have my set of data that I have split into the dependent variables (...
15 votes
3 answers
4k views

What is the proper way to use early stopping with cross-validation?

I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
1 vote
1 answer
222 views

Negative R2_score Bad predictions for my Sales prediction problem using LightGBM

My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows: <...
0 votes
1 answer
153 views

Model Dump Parser (like XGBFI) for LightGBM and CatBoost

Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
6 votes
1 answer
2k views

In XGBoost, how to change eval function and keeping same objective?

I would like to keep the objective as "reg:linear" and eval_metric as customized RMSE as follows: ...
0 votes
0 answers
24 views

XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression

i'm tring tune XGBoost model for my dataset. ...
0 votes
0 answers
29 views

Gradient tree boosting regularization term parameters

In the XGBoost documentation, they initially define the regularized objective as \begin{equation} \begin{split} L & = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i) + \sum\limits_{i=1}^K \Omega(f_k)\\...
0 votes
1 answer
14 views

Gradient tree boosting additive training

In the XGBoost documentation, they specify that the additive training is done given an objective $obj^{(t)}$ defined as $obj^{(t)} = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i^{(t-1)}+f_t(x_i)) + \sum\...
0 votes
2 answers
79 views

XgBoost given targets its only feature but fails when test targets are outside the range of training targets?

I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...
1 vote
2 answers
66 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
0 votes
1 answer
96 views

XGboost predict

I am trying to understand this XGboost example. After training: ptrain = bst.predict(dtrain, output_margin=True) they make prediction on test data, but the problem ...
3 votes
3 answers
68 views

Example for Boosting

Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
0 votes
2 answers
35 views

What is the implication of having features with less variation in a tree based model?

I'm training a tree-based model (e.g. xgb). I have some features with more than 90% values constant. Does it add value to the model since the variation in the data is minimal?. What would be the ...
1 vote
2 answers
78 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my model ...
0 votes
0 answers
27 views

Setting number of features in xgboost's DMatrix construction

The data argument in DMatrix's constructor (from xgboost library) supports giving path to a ...
1 vote
2 answers
263 views

Probabilities predicted by XGBoost

I looking at football data and trying to predict whether a goal will occur using xgboost with objective binary: logistic. My data is 1:10 unbalanced with no goals being more dominant. I have used ...
2 votes
1 answer
325 views

Can you reuse observations from your train data in your final test data?

For an employee population, I am trying to determine who among the employees are likely to get injured in the future based on 2 years worth of data. Unlike in most machine learning problems where you ...
1 vote
1 answer
88 views

Linear regression returning negative values for house price prediction

I am trying to do a prediction of real estate (prices are in millions). The mean price for the dataset is 4 million. I do not have any negative values in my dataset,...
0 votes
1 answer
744 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
38 votes
6 answers
41k views

Unbalanced multiclass data with XGBoost

I have 3 classes with this distribution: Class 0: 0.1169 Class 1: 0.7668 Class 2: 0.1163 And I am using xgboost for ...
5 votes
3 answers
213 views

How to use "tree boosting" with a data-driven loss function

We have a problem which has a data-driven (non-analytical) loss function. Our target contains whole numbers between 0 and 20 (the target is inherently discrete), although larger values are possible, ...
2 votes
1 answer
63 views

Learning to Rank with Unlabelled Dataset

I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
0 votes
0 answers
12 views

Time series forecasting model predicts increasing number for target variable when the actual values are zeroes

I am working on time series forecasting model, and I am using light gbm. The project goal is to predict the number of sales across different levels (very similar to the M5 competition). For instance, ...
0 votes
1 answer
282 views

XGBoost model has features whose feature importance equal zero

I ran into this problem: A XGBoost model(.pickle file , constrcuted under V0.7.post3) with 100 features in it ; But I found 55 features in model (model.feature_importances_) show 0 feature importance ...
1 vote
2 answers
896 views

On which algorithm for boosting is the method xgbLinear of the xgboost/caret- package based on?

In caret package of R, there is a method 'xgblinear'. What is the working algorithm behind this method.

1
2 3 4 5
13