Skip to main content

Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1 vote
0 answers
19 views

Multiclass matrix loss function in scikit-learn / xgboost / lightgbm

I have data with 4 classes: $c_1, c_2, c_3, c_4$. I'd like to create a classifier which has different scaling for the loss function per class combination: $$ \begin{bmatrix} 0 & l \left( \hat{c}_{...
Avi T's user avatar
  • 11
1 vote
1 answer
24 views

XGB predict_proba estimates don't match sum of leaves

When using an XGB model in the context of binary classification, I observed that the test estimates given by predict_proba were close but not equal to the results I ...
Juan Felipe Salamanca Lozano's user avatar
0 votes
0 answers
11 views

a few records for training, and one record after training

I try to make some credit score task. I stuck in conceputal problem. There is: train_data (62 columns, 10339239 rows, 1250000 unique ID values [0 - 1249999]([min-max] ID values)) test_data (62 ...
Excentricitet's user avatar
0 votes
0 answers
9 views

How does hyperparameter tuning work for constructing/choosing a final model using Nested Cross validation?

I want to determine if XGBoost is better than random forest or logistic regression for building a binary classification model. The model will be a composite model, with a feature selection model to ...
reuben george's user avatar
1 vote
0 answers
18 views

Weird Behavour in Skforcast

I applied XGBoost to forcast a univariante time series dataset, the first time I created my own lags features manually: ...
Bouabdallah khaled's user avatar
2 votes
0 answers
35 views

Tuning the learning rate parameter for GBDT models

I've always been taught that decreasing the learning rate parameter in gbdt models such as XGBoost, LightGBM and Catboost will improve the out-of-sample performance, assuming the number of iterations ...
Casper's user avatar
  • 21
0 votes
1 answer
21 views

How does deeper or shallower trees (higher or lower max_depth) affect xgboost model?

I am doing an xgboost model for landslides assessment and I am using max_depth as one of my hyperparameters, but I don't understand how does it affect model ...
Omab's user avatar
  • 3
0 votes
0 answers
16 views

What is the complexity of ICE (Individual conditional expectation) on ensemble trees

I'm evaluating different model-agnostic methods on gradient boosting or random forest model. For Shapley, specifically TreeSHAP, the complexity is O(TLD^2) according to Lundberg et al. 2018. T: #trees,...
wealthh2's user avatar
0 votes
0 answers
23 views

Difference in the value of evaluation-metric in xgb.train() and predict in R

I have trained a xgboost classifier with a custom metric (f1_xgb), that is, the F1 score. Here the important aspect is that I evaluated the classifier on the test set by setting: ...
user159339's user avatar
0 votes
0 answers
44 views

Gradient function in LogisitcLoss class

I am going through a code for XGBoost from scratch and I am referring to this repository here The log-loss function is given by On differentiating the above function with respect to y_pred (referring ...
Mehul Jain's user avatar
0 votes
1 answer
34 views

How is the weight of each new weak learner is calculated in Xgboost?

In Xgboost we have multiple sequential weak learner. Let say I have weak learner WL1 and we fitted it on our data and we calulated the error. Now we have another weak learner WL2. And as I have read ...
XGB's user avatar
  • 15
0 votes
2 answers
292 views

Handling categorical variables for Xgboost?

Currently there seems to be two approaches for handling categorical variables in gbdts: Xgboost as an option, but data need to be encoded properly (integers) Catboost can handle everything provided ...
Lucas Morin's user avatar
  • 2,196
0 votes
1 answer
86 views

Xgboost multiclass monotonic constraints

I have a problem where i have a variable price and i need to classify this price as winning/non-winning. If price grows, probability should monotonically go down. I use a monotonic constraint that ...
Jose Cle's user avatar
0 votes
0 answers
74 views

How to do feature selection correctly in xgboost for time series forecasting after obtaining a good predictive model?

I have a very large dataset (~7 million rows) for which I have extracted ~500 features during feature engineering phase. I have trained an XGBoost which has a fairly good predictive capability (based ...
guestar's user avatar
0 votes
0 answers
21 views

LightGBM Regressor miscalibratred/underestimating on high fitted values and overestimating on low fitted values

I'm training a pretty standard LightGBM regressor and noticing a strange pattern with the residuals (see images below--I'm bunching the predicted values and taking the observed average for the group). ...
dfried's user avatar
  • 101
0 votes
0 answers
60 views

How to deal with missing values in the output for XGBoost Regressor

So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value. This is because ...
RM25's user avatar
  • 1
0 votes
0 answers
58 views

Custom objective function for xgboost to optimize lift in best decile

I've tried to define a custom objective function for xgboost to optimize the lift for a binary classification problem in the upper decile. The task is simply to concentrate the training effort to the ...
Arne Bøckmann's user avatar
0 votes
1 answer
155 views

Understand and compute confidence interval and coefficient of variation for regression model

I would like to better understand the concepts of: coefficient of variation and confidence interval. Trivially taking the definitions from wikipedia: confidence interval (CI) In frequentist ...
Cata's user avatar
  • 1
0 votes
0 answers
57 views

Ordinal log-loss in a multiclass classification in XGBoost?

I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High). What I would ...
deblue's user avatar
  • 111
0 votes
0 answers
143 views

Making ML Model (XGBoost) Output Smoother

I am using XGBoost for a classification problem. The model has multiple inputs and a probability as output. For simplicity's sake, let's say the model only has one continous input and the function to ...
Neo's user avatar
  • 11
0 votes
0 answers
7 views

Histogram creation in lightgbm in the train API and the scikit-learn API. Is it always benefitial to use the train API?

In the LightGBM for python we have a scikit-learn API in which (either for regression or for classification) there is fit method whose documentation is fit(X, y, sample_weight=None, init_score=None, ...
figs_and_nuts's user avatar
0 votes
0 answers
59 views

Xgboost giving different results on windows and Linux

I'm facing one issue with the results not matching on Windows and Linux. I'm using the same code on both machines. Using separate trains. csv and test. csv The problem is due to sampling and ...
Sagar J's user avatar
0 votes
1 answer
41 views

Validity of using raw time series data for training of xgboost/random forest classifier

I am currently working on a project aiming of classification of process states based on time series data. For this, we are looking at different models, such as XGBoost-based classifiers or ...
arc_lupus's user avatar
  • 103
0 votes
0 answers
18 views

Minimize MAE loss for a target that is sum of two other targets

Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
Ngo Cuong's user avatar
0 votes
0 answers
24 views

Binary classification using xgboost

Why when adding new features in my ADS for a binary classification using XGBOOST my score and uplift has decreased ? What is the best way to treate categorical features or other features in order that ...
Warda_IDRIS's user avatar
0 votes
0 answers
19 views

Parameter outside specified range using HGboost/Hyperopt library

I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
AWGIS's user avatar
  • 101
1 vote
2 answers
131 views

Xgboost model predicting extreme values for events and non-events | Overfitting

Extreme values are predicted by my trained xgboost classification model in BQML for both events (Y=1) and non-events (Y=0). For all event observations, the model calculates probability scores that ...
Scott Grammilo's user avatar
0 votes
0 answers
42 views

XGBoost Classifier Evaluation Confusion on New Dataset Despite High Cross-Validation Scores

I have built an XGBoost classifier model with 90 features, trained on a dataset containing 760k samples. I took great care to separate the labels from the features in both the training and testing ...
oklen's user avatar
  • 1
1 vote
0 answers
59 views

Understanding lgbm histogram building

...
figs_and_nuts's user avatar
0 votes
0 answers
67 views

XGBoost Classifier + Isotonic Regression leading to worse probability accuracy

I'm testing out an XGBoost Classifier with the goal of using the probabilities it predicts in production. I know that tree based model probabilities are often not calibrated well so I decided to test ...
Ted's user avatar
  • 1
0 votes
0 answers
270 views

XGBoost tweedie regression objetive from scratch

I'm trying to gain a deeper understanding of the tweedie loss function and how it is used in XGBoost. So, I tried to implement it from scratch. I started by examining the original implementation. I ...
Diadochokinetic's user avatar
0 votes
0 answers
62 views

XGBRegressor underestimates sum of regression target (insurance application)

I'm trying to learn how to apply Boostig Algorithms (e.g. XGBoost) to insurance applications (premium calculation). As a starting point I used this tutorial from the scikit-learn website. tldr: The ...
Diadochokinetic's user avatar
1 vote
1 answer
199 views

Quality of first tree prediction in xgboost

Up until now I have mostly been using neural nets for regression. There on each iteration (epoch) the weights are updated using the whole training data, and we expect the error to go down each time, ...
SBF's user avatar
  • 111
0 votes
0 answers
72 views

Problem calculating f-1 score when when running hyperparameter tuning using Optuna

I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
Metrician's user avatar
1 vote
0 answers
72 views

Replication of XGBoost's binary:logistic loss

I am trying to replicate XGBoost's logistic loss function as a first step before implementing my own custom loss functions. Following from here and looking at the original code in git repository, I ...
deblue's user avatar
  • 111
1 vote
1 answer
73 views

Time series prediction on test set produces multiple cycles instead of following trend

I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use. There's just one feature ...
Thermal_insulator's user avatar
0 votes
1 answer
112 views

XGBoost with objective as binary:logistic returning only binary value

I am training an xgboost model for binary classification using objective as 'binary:logistic'. The model should predict probability but it is outputting either 0 or 1. I want the model to output the ...
shivani's user avatar
  • 140
1 vote
3 answers
161 views

Feature importance score for a feature that contains mostly 0's in XGBoost

I have read that the feature importance scores are calculated based on how a split on that feature improves performance. I have a binary classification dataset and am running XGBoost classifier on it. ...
Vjs's user avatar
  • 13
0 votes
1 answer
69 views

eval_metric of XGBoost // ML model in general

Say I am using Xgboost on a binary classification task. eval_metric is one of the model parameter. How should I think about the impact of using different eval_metric(e.g rmse/mae/logloss) in general? ...
pathtoagi's user avatar
0 votes
0 answers
89 views

Can I decompose SHAP interaction values like a linear regression?

I had a question regarding the shap interaction matrix. Suppose I have 500 samples with 2 features. Then my interaction matrix becomes (500,2,2). I want to calculate the SHAP values of each feature ...
cwanderroycbooks's user avatar
0 votes
0 answers
54 views

Build a model with cross-validation on entire dataset to learn insights?

Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed. Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
cwanderroycbooks's user avatar
0 votes
0 answers
67 views

XGBoost Architecture Diagram required

Good Day! My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
P_Z's user avatar
  • 101
0 votes
0 answers
55 views

weight derivation for XGBOOST in regression trees

I am working on a variation of the XGBOOST presented on the following paper by Chen and Guestrin: https://dl.acm.org/doi/abs/10.1145/2939672.2939785 My goal is to fit a linear regression in each leaf ...
firstname's user avatar
0 votes
0 answers
67 views

How to predict an xgboost model outcome directly from the trained trees?

I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it ...
Subhajit Saha's user avatar
1 vote
1 answer
186 views

Funny looking learning curve

I am training an XGBoost for binary classification. I am getting a very odd-looking learning curve. Any idea what is going on and how I might fix this? Thanks
pyassign67's user avatar
1 vote
1 answer
360 views

Why is accuracy score suddenly becoming 1 on using XGBoost?

I am developing a music classification system based on a kaggle dataset: https://www.kaggle.com/datasets/vatsalmavani/spotify-dataset I tried using K means classifier to classify the songs into 4 ...
zero_day's user avatar
0 votes
0 answers
48 views

Why am I getting differing "gain" feature importances from XGBoost?

I am currently training an XGBoost model for binary classification. I have fitted and predicted with the model but when I try to get the "gain" type feature importances, the results differ ...
user19941466's user avatar
0 votes
0 answers
272 views

How do the predict() and predict_proba() for xgbClassifier() work?

I am using xgboost classifier for a binary classification problem. I used the predict() to get the class predictions (0/1) but I also used the predict_proba() method to get the class probabilities (my ...
user62198's user avatar
  • 1,091
0 votes
1 answer
331 views

How to intrepret low F1 score and high AUC on training set?

I am currently working on a very imbalanced dataset: 24 million transactions (rows of data) 30,000 fraudulent transactions (0.1% of total transactions) and I am using XGBoost as the model to predict ...
Hai Nguyen's user avatar
0 votes
0 answers
59 views

Why does my accuracy score drop after hyperparameter tuning in XGBoost?

I am trying to tune the model I've built, but every time I change hyperparameters my accuracy score drops significantly. I'm using RandomizedSearchCV and best_params_ to determine which parameters I ...
mvinegret's user avatar

1
2 3 4 5
15