Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0 votes
1 answer
111 views

Predicted and true values distributions comparison

Is this alarming when a distribution of predicted values differs from a distribution of true values? I use xgbregressor and get the following plots Usage of boxcox doesn't improve the case. My data ...
James Flash's user avatar
0 votes
1 answer
1k views

Adding time as a feature with xgboost/random forests

I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
Luca's user avatar
  • 183
0 votes
3 answers
2k views

Specifying number of threads using XGBoost.train

When using the xgboost.train() function, all the threads are used. I would like to use a specific amount. Unfortunately, this function does not accept the ...
LauritsT's user avatar
0 votes
2 answers
612 views

Xgboost take k best predictions

I have a mission of classification with a lot of classes. I am comparing some ML algorithms for this case and I would like to try xgboost. I am writing in python and I am trying to get the best 3 ...
Ariel Amar's user avatar
0 votes
1 answer
904 views

How to forecast product/item sales next one week using Xgboost regressor

I have trained xgboost algorithm to predict the number of items sale on a given day and got pretty good results, now I would like to forecast sales ahead of one week. I tried re-training the ...
Optimizor's user avatar
0 votes
1 answer
226 views

How to interpret a random variable in the variable importance?

I have a problem, for simplicity let's say it is a binary classification problem. I am trying to solve this problem using XGBoost. A standard output plot for any ML algorithm, is the feature ...
quant's user avatar
  • 353
0 votes
4 answers
142 views

Can this problem be solved using deep learning?

I want to predict price of used cars. I have data like this: Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.? I used one hot approach for nominal features and ...
arslanbenzer's user avatar
0 votes
1 answer
268 views

XGBOOST CV() producing error

I am getting the following error while using xgboost.cv() (scikit-learn interface). I am working on a regression problem. Below is the code and trace. No idea why ...
user62198's user avatar
  • 1,091
0 votes
1 answer
74 views

xgboost and linear regression new feature analysis

For linear regression, seems like a new feature has to be a linear relation with the target variable. But If you make the new feature for the Xgboost, what do you have to consider to make a new ...
slowmonk's user avatar
  • 513
0 votes
1 answer
1k views

How to favour a particular class during classification using XGBoost?

I am using a simple XGBoost model to classify 2 classes (0 and 1) in a binary context. In case of the original data, the 0 is the majority class and 1 the minority class. The thing which is happening ...
JChat's user avatar
  • 197
0 votes
1 answer
133 views

python: why add new features of image to be worse result with xgboost

there are about 93 rows of data with features, two classes label. And, there are about 49 one-hot value features, and there are about 10 features continuous value. I split the data randomly by train ...
tktktk0711's user avatar
0 votes
1 answer
162 views

XGBoost: predictive, descriptive (or both) model?

I have trained an XGBoost model for prediction. The algorithm is able to calculate variable importances. I was asked why I have not analyzed these variable importances and I did not because as I ...
aqua's user avatar
  • 123
0 votes
1 answer
47 views

The best w_j confusion in xgboost

from XGBoost tutorial, it described: In this equation $w_j$ are independent with respect to each other, the form $G_j w_j + \frac{1}{2}(H_j+λ)w_j^2$ is quadratic and the best $w_j$ for a given ...
cinqS's user avatar
  • 367
0 votes
1 answer
466 views

Do xgboost and random forests in general handle multiple splits of the same numeric feature in a single branch?

For example, let's say that the age (say $x$) of a person for $12< x< 25$ can be used to predict computer usage to a high degree of certainty. In a decision tree, this could be represented by a ...
Frustrated_Student's user avatar
0 votes
1 answer
376 views

XGBClassifier and RandomForestClassifier variable importance plots are very different

I have one dataset, and decided to use XGBClassifier to get a variable importance plot from it. I was a little surprised that some features that I assumed were insightful had no appreciable value. ...
Monica Heddneck's user avatar
0 votes
1 answer
2k views

Dynamic learning rates in XGBoost cross-validation

XGBoost's xgb.train() method takes a learning_rates parameter, which can take a custom function to apply a dynamic learning rate,...
Jivan's user avatar
  • 165
0 votes
0 answers
10 views

a few records for training, and one record after training

I try to make some credit score task. I stuck in conceputal problem. There is: train_data (62 columns, 10339239 rows, 1250000 unique ID values [0 - 1249999]([min-max] ID values)) test_data (62 ...
Excentricitet's user avatar
0 votes
0 answers
9 views

How does hyperparameter tuning work for constructing/choosing a final model using Nested Cross validation?

I want to determine if XGBoost is better than random forest or logistic regression for building a binary classification model. The model will be a composite model, with a feature selection model to ...
reuben george's user avatar
0 votes
1 answer
21 views

How does deeper or shallower trees (higher or lower max_depth) affect xgboost model?

I am doing an xgboost model for landslides assessment and I am using max_depth as one of my hyperparameters, but I don't understand how does it affect model ...
Omab's user avatar
  • 3
0 votes
0 answers
16 views

What is the complexity of ICE (Individual conditional expectation) on ensemble trees

I'm evaluating different model-agnostic methods on gradient boosting or random forest model. For Shapley, specifically TreeSHAP, the complexity is O(TLD^2) according to Lundberg et al. 2018. T: #trees,...
wealthh2's user avatar
0 votes
0 answers
23 views

Difference in the value of evaluation-metric in xgb.train() and predict in R

I have trained a xgboost classifier with a custom metric (f1_xgb), that is, the F1 score. Here the important aspect is that I evaluated the classifier on the test set by setting: ...
user159339's user avatar
0 votes
0 answers
44 views

Gradient function in LogisitcLoss class

I am going through a code for XGBoost from scratch and I am referring to this repository here The log-loss function is given by On differentiating the above function with respect to y_pred (referring ...
Mehul Jain's user avatar
0 votes
0 answers
72 views

How to do feature selection correctly in xgboost for time series forecasting after obtaining a good predictive model?

I have a very large dataset (~7 million rows) for which I have extracted ~500 features during feature engineering phase. I have trained an XGBoost which has a fairly good predictive capability (based ...
guestar's user avatar
0 votes
0 answers
21 views

LightGBM Regressor miscalibratred/underestimating on high fitted values and overestimating on low fitted values

I'm training a pretty standard LightGBM regressor and noticing a strange pattern with the residuals (see images below--I'm bunching the predicted values and taking the observed average for the group). ...
dfried's user avatar
  • 101
0 votes
0 answers
60 views

How to deal with missing values in the output for XGBoost Regressor

So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value. This is because ...
RM25's user avatar
  • 1
0 votes
0 answers
58 views

Custom objective function for xgboost to optimize lift in best decile

I've tried to define a custom objective function for xgboost to optimize the lift for a binary classification problem in the upper decile. The task is simply to concentrate the training effort to the ...
Arne Bøckmann's user avatar
0 votes
0 answers
56 views

Ordinal log-loss in a multiclass classification in XGBoost?

I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High). What I would ...
deblue's user avatar
  • 111
0 votes
0 answers
141 views

Making ML Model (XGBoost) Output Smoother

I am using XGBoost for a classification problem. The model has multiple inputs and a probability as output. For simplicity's sake, let's say the model only has one continous input and the function to ...
Neo's user avatar
  • 11
0 votes
0 answers
7 views

Histogram creation in lightgbm in the train API and the scikit-learn API. Is it always benefitial to use the train API?

In the LightGBM for python we have a scikit-learn API in which (either for regression or for classification) there is fit method whose documentation is fit(X, y, sample_weight=None, init_score=None, ...
figs_and_nuts's user avatar
0 votes
0 answers
57 views

Xgboost giving different results on windows and Linux

I'm facing one issue with the results not matching on Windows and Linux. I'm using the same code on both machines. Using separate trains. csv and test. csv The problem is due to sampling and ...
Sagar J's user avatar
0 votes
0 answers
18 views

Minimize MAE loss for a target that is sum of two other targets

Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
Ngo Cuong's user avatar
0 votes
0 answers
24 views

Binary classification using xgboost

Why when adding new features in my ADS for a binary classification using XGBOOST my score and uplift has decreased ? What is the best way to treate categorical features or other features in order that ...
Warda_IDRIS's user avatar
0 votes
0 answers
18 views

Parameter outside specified range using HGboost/Hyperopt library

I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
AWGIS's user avatar
  • 101
0 votes
0 answers
40 views

XGBoost Classifier Evaluation Confusion on New Dataset Despite High Cross-Validation Scores

I have built an XGBoost classifier model with 90 features, trained on a dataset containing 760k samples. I took great care to separate the labels from the features in both the training and testing ...
oklen's user avatar
  • 1
0 votes
0 answers
64 views

XGBoost Classifier + Isotonic Regression leading to worse probability accuracy

I'm testing out an XGBoost Classifier with the goal of using the probabilities it predicts in production. I know that tree based model probabilities are often not calibrated well so I decided to test ...
Ted's user avatar
  • 1
0 votes
0 answers
262 views

XGBoost tweedie regression objetive from scratch

I'm trying to gain a deeper understanding of the tweedie loss function and how it is used in XGBoost. So, I tried to implement it from scratch. I started by examining the original implementation. I ...
Diadochokinetic's user avatar
0 votes
0 answers
59 views

XGBRegressor underestimates sum of regression target (insurance application)

I'm trying to learn how to apply Boostig Algorithms (e.g. XGBoost) to insurance applications (premium calculation). As a starting point I used this tutorial from the scikit-learn website. tldr: The ...
Diadochokinetic's user avatar
0 votes
0 answers
70 views

Problem calculating f-1 score when when running hyperparameter tuning using Optuna

I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
Metrician's user avatar
0 votes
0 answers
88 views

Can I decompose SHAP interaction values like a linear regression?

I had a question regarding the shap interaction matrix. Suppose I have 500 samples with 2 features. Then my interaction matrix becomes (500,2,2). I want to calculate the SHAP values of each feature ...
cwanderroycbooks's user avatar
0 votes
0 answers
54 views

Build a model with cross-validation on entire dataset to learn insights?

Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed. Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
cwanderroycbooks's user avatar
0 votes
0 answers
67 views

XGBoost Architecture Diagram required

Good Day! My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
P_Z's user avatar
  • 101
0 votes
0 answers
54 views

weight derivation for XGBOOST in regression trees

I am working on a variation of the XGBOOST presented on the following paper by Chen and Guestrin: https://dl.acm.org/doi/abs/10.1145/2939672.2939785 My goal is to fit a linear regression in each leaf ...
firstname's user avatar
0 votes
0 answers
65 views

How to predict an xgboost model outcome directly from the trained trees?

I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it ...
Subhajit Saha's user avatar
0 votes
0 answers
48 views

Why am I getting differing "gain" feature importances from XGBoost?

I am currently training an XGBoost model for binary classification. I have fitted and predicted with the model but when I try to get the "gain" type feature importances, the results differ ...
user19941466's user avatar
0 votes
0 answers
266 views

How do the predict() and predict_proba() for xgbClassifier() work?

I am using xgboost classifier for a binary classification problem. I used the predict() to get the class predictions (0/1) but I also used the predict_proba() method to get the class probabilities (my ...
user62198's user avatar
  • 1,091
0 votes
0 answers
59 views

Why does my accuracy score drop after hyperparameter tuning in XGBoost?

I am trying to tune the model I've built, but every time I change hyperparameters my accuracy score drops significantly. I'm using RandomizedSearchCV and best_params_ to determine which parameters I ...
mvinegret's user avatar
0 votes
0 answers
87 views

Visualize Catboost and XGBoost training process + Cross Validation

I want to optimize Catboost and XGBoost models and visualize this process such that: Use 3-fold cross-validation Use my own pre-processing pipeline (Missing value imputation, over- or undersampling) ...
Ars ML's user avatar
  • 61
0 votes
0 answers
16 views

Data processing - how to input a pandas column that contains numbers and numpy arrays

I have a pandas df that contains numbers and strings. I use word2vec to convert all the strings into embeddings. The problem now is that these embeddings are all numpy arrays. So now my pandas df ...
cyc's user avatar
  • 1
0 votes
1 answer
33 views

Validating ML regression model and predictions

I have a years worth of electricity power data on 15 minute intervals joined with weather data and time-of-week one hot dummy variables. Is using train/test split an okay approach for validating the ...
bbartling's user avatar
  • 403
0 votes
1 answer
681 views

XGB DataFrame.dtypes error

Here is the code to assign the variables ...
Nata's user avatar
  • 3