Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
704
questions
0
votes
1
answer
111
views
Predicted and true values distributions comparison
Is this alarming when a distribution of predicted values differs from a distribution of true values? I use xgbregressor and get the following plots
Usage of boxcox doesn't improve the case.
My data ...
0
votes
1
answer
1k
views
Adding time as a feature with xgboost/random forests
I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
0
votes
3
answers
2k
views
Specifying number of threads using XGBoost.train
When using the xgboost.train() function, all the threads are used. I would like to use a specific amount. Unfortunately, this function does not accept the ...
0
votes
2
answers
612
views
Xgboost take k best predictions
I have a mission of classification with a lot of classes. I am comparing some ML algorithms for this case and I would like to try xgboost.
I am writing in python and I am trying to get the best 3 ...
0
votes
1
answer
904
views
How to forecast product/item sales next one week using Xgboost regressor
I have trained xgboost algorithm to predict the number of items sale on a given day and got pretty good results, now I would like to forecast sales ahead of one week.
I tried re-training the ...
0
votes
1
answer
226
views
How to interpret a random variable in the variable importance?
I have a problem, for simplicity let's say it is a binary classification problem.
I am trying to solve this problem using XGBoost.
A standard output plot for any ML algorithm, is the feature ...
0
votes
4
answers
142
views
Can this problem be solved using deep learning?
I want to predict price of used cars. I have data like this:
Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.?
I used one hot approach for nominal features and ...
0
votes
1
answer
268
views
XGBOOST CV() producing error
I am getting the following error while using xgboost.cv() (scikit-learn interface). I am working on a regression problem. Below is the code and trace. No idea why ...
0
votes
1
answer
74
views
xgboost and linear regression new feature analysis
For linear regression, seems like a new feature has to be a linear relation with the target variable.
But If you make the new feature for the Xgboost, what do you have to consider to make a new ...
0
votes
1
answer
1k
views
How to favour a particular class during classification using XGBoost?
I am using a simple XGBoost model to classify 2 classes (0 and 1) in a binary context. In case of the original data, the 0 is the majority class and 1 the minority class. The thing which is happening ...
0
votes
1
answer
133
views
python: why add new features of image to be worse result with xgboost
there are about 93 rows of data with features, two classes label. And, there are about 49 one-hot value features, and there are about 10 features continuous value. I split the data randomly by train ...
0
votes
1
answer
162
views
XGBoost: predictive, descriptive (or both) model?
I have trained an XGBoost model for prediction. The algorithm is able to calculate variable importances. I was asked why I have not analyzed these variable importances and I did not because as I ...
0
votes
1
answer
47
views
The best w_j confusion in xgboost
from XGBoost tutorial, it described:
In this equation $w_j$ are independent with respect to each other, the form $G_j w_j + \frac{1}{2}(H_j+λ)w_j^2$ is quadratic and the best $w_j$ for a given ...
0
votes
1
answer
466
views
Do xgboost and random forests in general handle multiple splits of the same numeric feature in a single branch?
For example, let's say that the age (say $x$) of a person for $12< x< 25$ can be used to predict computer usage to a high degree of certainty. In a decision tree, this could be represented by a ...
0
votes
1
answer
376
views
XGBClassifier and RandomForestClassifier variable importance plots are very different
I have one dataset, and decided to use XGBClassifier to get a variable importance plot from it.
I was a little surprised that some features that I assumed were insightful had no appreciable value. ...
0
votes
1
answer
2k
views
Dynamic learning rates in XGBoost cross-validation
XGBoost's xgb.train() method takes a learning_rates parameter, which can take a custom function to apply a dynamic learning rate,...
0
votes
0
answers
10
views
a few records for training, and one record after training
I try to make some credit score task. I stuck in conceputal problem.
There is:
train_data (62 columns, 10339239 rows, 1250000 unique ID values [0 - 1249999]([min-max] ID values))
test_data (62 ...
0
votes
0
answers
9
views
How does hyperparameter tuning work for constructing/choosing a final model using Nested Cross validation?
I want to determine if XGBoost is better than random forest or logistic regression for building a binary classification model. The model will be a composite model, with a feature selection model to ...
0
votes
1
answer
21
views
How does deeper or shallower trees (higher or lower max_depth) affect xgboost model?
I am doing an xgboost model for landslides assessment and I am using max_depth as one of my hyperparameters, but I don't understand how does it affect model ...
0
votes
0
answers
16
views
What is the complexity of ICE (Individual conditional expectation) on ensemble trees
I'm evaluating different model-agnostic methods on gradient boosting or random forest model.
For Shapley, specifically TreeSHAP, the complexity is O(TLD^2) according to Lundberg et al. 2018. T: #trees,...
0
votes
0
answers
23
views
Difference in the value of evaluation-metric in xgb.train() and predict in R
I have trained a xgboost classifier with a custom metric (f1_xgb), that is, the F1 score. Here the important aspect is that I evaluated the classifier on the test set by setting:
...
0
votes
0
answers
44
views
Gradient function in LogisitcLoss class
I am going through a code for XGBoost from scratch and I am referring to this repository here
The log-loss function is given by
On differentiating the above function with respect to y_pred (referring ...
0
votes
0
answers
72
views
How to do feature selection correctly in xgboost for time series forecasting after obtaining a good predictive model?
I have a very large dataset (~7 million rows) for which I have extracted ~500 features during feature engineering phase. I have trained an XGBoost which has a fairly good predictive capability (based ...
0
votes
0
answers
21
views
LightGBM Regressor miscalibratred/underestimating on high fitted values and overestimating on low fitted values
I'm training a pretty standard LightGBM regressor and noticing a strange pattern with the residuals (see images below--I'm bunching the predicted values and taking the observed average for the group). ...
0
votes
0
answers
60
views
How to deal with missing values in the output for XGBoost Regressor
So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value.
This is because ...
0
votes
0
answers
58
views
Custom objective function for xgboost to optimize lift in best decile
I've tried to define a custom objective function for xgboost to optimize the lift for a binary classification problem in the upper decile. The task is simply to concentrate the training effort to the ...
0
votes
0
answers
56
views
Ordinal log-loss in a multiclass classification in XGBoost?
I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High).
What I would ...
0
votes
0
answers
141
views
Making ML Model (XGBoost) Output Smoother
I am using XGBoost for a classification problem. The model has multiple inputs and a probability as output.
For simplicity's sake, let's say the model only has one continous input and the function to ...
0
votes
0
answers
7
views
Histogram creation in lightgbm in the train API and the scikit-learn API. Is it always benefitial to use the train API?
In the LightGBM for python we have a scikit-learn API in which (either for regression or for classification) there is fit method whose documentation is
fit(X, y, sample_weight=None, init_score=None, ...
0
votes
0
answers
57
views
Xgboost giving different results on windows and Linux
I'm facing one issue with the results not matching on Windows and Linux. I'm using the same code on both machines. Using separate trains. csv and test. csv
The problem is due to sampling and ...
0
votes
0
answers
18
views
Minimize MAE loss for a target that is sum of two other targets
Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
0
votes
0
answers
24
views
Binary classification using xgboost
Why when adding new features in my ADS for a binary classification using XGBOOST my score and uplift has decreased ? What is the best way to treate categorical features or other features in order that ...
0
votes
0
answers
18
views
Parameter outside specified range using HGboost/Hyperopt library
I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
0
votes
0
answers
40
views
XGBoost Classifier Evaluation Confusion on New Dataset Despite High Cross-Validation Scores
I have built an XGBoost classifier model with 90 features, trained on a dataset containing 760k samples. I took great care to separate the labels from the features in both the training and testing ...
0
votes
0
answers
64
views
XGBoost Classifier + Isotonic Regression leading to worse probability accuracy
I'm testing out an XGBoost Classifier with the goal of using the probabilities it predicts in production. I know that tree based model probabilities are often not calibrated well so I decided to test ...
0
votes
0
answers
262
views
XGBoost tweedie regression objetive from scratch
I'm trying to gain a deeper understanding of the tweedie loss function and how it is used in XGBoost. So, I tried to implement it from scratch. I started by examining the original implementation.
I ...
0
votes
0
answers
59
views
XGBRegressor underestimates sum of regression target (insurance application)
I'm trying to learn how to apply Boostig Algorithms (e.g. XGBoost) to insurance applications (premium calculation). As a starting point I used this tutorial from the scikit-learn website.
tldr: The ...
0
votes
0
answers
70
views
Problem calculating f-1 score when when running hyperparameter tuning using Optuna
I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
0
votes
0
answers
88
views
Can I decompose SHAP interaction values like a linear regression?
I had a question regarding the shap interaction matrix.
Suppose I have 500 samples with 2 features. Then my interaction matrix becomes (500,2,2).
I want to calculate the SHAP values of each feature ...
0
votes
0
answers
54
views
Build a model with cross-validation on entire dataset to learn insights?
Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed.
Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
0
votes
0
answers
67
views
XGBoost Architecture Diagram required
Good Day!
My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
0
votes
0
answers
54
views
weight derivation for XGBOOST in regression trees
I am working on a variation of the XGBOOST presented on the following paper by Chen and Guestrin: https://dl.acm.org/doi/abs/10.1145/2939672.2939785
My goal is to fit a linear regression in each leaf ...
0
votes
0
answers
65
views
How to predict an xgboost model outcome directly from the trained trees?
I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it ...
0
votes
0
answers
48
views
Why am I getting differing "gain" feature importances from XGBoost?
I am currently training an XGBoost model for binary classification. I have fitted and predicted with the model but when I try to get the "gain" type feature importances, the results differ ...
0
votes
0
answers
266
views
How do the predict() and predict_proba() for xgbClassifier() work?
I am using xgboost classifier for a binary classification problem. I used the predict() to get the class predictions (0/1) but I also used the predict_proba() method to get the class probabilities (my ...
0
votes
0
answers
59
views
Why does my accuracy score drop after hyperparameter tuning in XGBoost?
I am trying to tune the model I've built, but every time I change hyperparameters my accuracy score drops significantly. I'm using RandomizedSearchCV and best_params_ to determine which parameters I ...
0
votes
0
answers
87
views
Visualize Catboost and XGBoost training process + Cross Validation
I want to optimize Catboost and XGBoost models and visualize this process such that:
Use 3-fold cross-validation
Use my own pre-processing pipeline (Missing value imputation, over- or undersampling)
...
0
votes
0
answers
16
views
Data processing - how to input a pandas column that contains numbers and numpy arrays
I have a pandas df that contains numbers and strings. I use word2vec to convert all the strings into embeddings. The problem now is that these embeddings are all numpy arrays. So now my pandas df ...
0
votes
1
answer
33
views
Validating ML regression model and predictions
I have a years worth of electricity power data on 15 minute intervals joined with weather data and time-of-week one hot dummy variables.
Is using train/test split an okay approach for validating the ...
0
votes
1
answer
681
views
XGB DataFrame.dtypes error
Here is the code to assign the variables
...