Data Science

Questions tagged [xgboost]

Ask Question

For questions related to the eXtreme Gradient Boosting algorithm.

344 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
1
vote
0answers
10 views

best way to regularize gradient boosting regressor?

i am testing gradient boosting regressor from sklearn for time series prediction on noisy data (currency markets). https://scikit-learn.org/stable/modules/generated/sklearn.ensemble....
0
votes
1answer
10 views

sklearn.feature_selection vs xgboost feature_importances?

sklearn.feature_selection vs xgboost feature_importances Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
0
votes
0answers
18 views

Why XGBoost regressor predicts behavior but not the amplitude?

I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling). I found out that it works very well for predicting the ...
1
vote
1answer
30 views

Average of importance gain for a categorical variable

Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc). I train an XGBoost model on a numeric ...
1
vote
1answer
24 views

XGBClassifier: make the output of predict_proba ascending regarding a specific feature

I'm trying to build a classifier using Xgboost on some high dimensional data, the problem I'm having is that I have the prior knowledge that the output probabilities should be ascending regarding a ...
0
votes
0answers
17 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
1
vote
0answers
13 views

SHAP Explanations in case of repeated train/test split

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that ...
1
vote
2answers
21 views

Adding extra variables to XGboost model is worsening the train and test accuracy

I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split. However, when I am adding two new features , the ...
1
vote
0answers
19 views

Feature Vectors representation

I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
0
votes
0answers
33 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
1
vote
0answers
41 views

Hyperparameter Tuning using Bayesian Techniques

I've been looking into Bayesian optimization for hyperparameter tuning and trying to compare the results I get to those I get using different methods (random grid search). I came across this site, ...
2
votes
0answers
28 views

Can I specify the root node splitting feature in XGBoost?

Just what the title says. Suppose I know the feature that I want to be used for splitting for the root node in a tree model in XGBoost; is there a way for me to tell XGBoost to split on this feature ...
0
votes
0answers
18 views

Feature engineering ideas with dates, coordinates and other variables

I'm working on an ETA problem where I'm trying to estimate a time of arrival for a delivery. I have coordinates of pickup/destination, time of pick , infos about the rider, some other variables that ...
0
votes
0answers
64 views

Suggestion for handling specific missing data

I have data, that describes distance from given location to nearest object (e.g. school, shop etc). Because of performance reasons I couldn't scrape the data about objects, that are futher away than 2....
0
votes
0answers
10 views

Improve the results of imbalanced multi-classification multi-lables data

I have 10k rows of multi-classification (x1..x27,y), size of dataframe is: 28*10k and its ...
4
votes
2answers
73 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...
1
vote
1answer
36 views

How does XGBoost use softmax as an objective function?

I'm quite used to seeing functions like log-loss, RMSE, cross entropy as objective functions and it's easy to imagine why minimizing these would give us the best model. What's difficult to imagine is ...
0
votes
2answers
59 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
0
votes
2answers
85 views

What is the difference between gradient descent and gradient boosting? Are they interdependent on each other by any way?

What is the difference between gradient descent and gradient boosting? Are they interdependent on each other in any way ?
0
votes
1answer
26 views

XGBoost probability distribution tending towards the extreme

I am using an XGBoost classifier to make risk predictions, and I see that even if it has very good binary classification results, the probability outputs are mainly under $0.05$ or over $0.95$ (like ...
0
votes
1answer
25 views

Similarity of XGBoost models?

Is xgboost with n_estimators = 100 and learning_rate = 0.1, same as xgboost with n_estimators = 50 and learning_rate = 0.2 ?
1
vote
0answers
26 views

XGBoost speed issues

I'm trying to optimize the hyperparameters for XGBoost, thus needing to run it multiple times with different parameters. However the time needed to run single XGBoost with the parameters provided ...
0
votes
0answers
16 views

Data Transformation Tips for xgboost's XGBClassifier

I have this X_train and test distribution for the 4 features 'X', 'Y', 'TX' and 'TY'. I realize the range of the distribution is widely varying .. Can you suggest a good way to clean/ transform that ...
3
votes
2answers
151 views

XGBoost validation for number of trees

I have a simple Question: I am using XGBoost to classify some data: 1.) With 100 estimators I have following scores(roc_score): train_data : 98.5 validation_data : 97.2 2.) With 500 ...
0
votes
1answer
19 views

How to tell a boosting model that 2 features are related and should not be interpreted stand-alone?

I am using XGBoost for a machine learning model that learns from tabular data. XGBoost uses boosting method on decision trees. When I look at the decision-making logic of decision trees, I notice the ...
1
vote
1answer
23 views

xgboost in R have different results compared to boosted decision tree in Azure ML

I have a small data set (4000 records with 10 features) and I used XGBOOST in R as well as Boosted Decision Tree model in Azure ML studio. Unfortunately the results are different. I like to optimize ...
4
votes
1answer
114 views

Forecasting via LSTM or XGBoost… is it really a forecast or

I guess I understand the idea of predictions made via LSTM or XGBoost models, but want to reach out to the community to confirm my thoughts. This tutorial does a nice job explaining step by step of ...
0
votes
0answers
25 views

if I got feature importance of xgboost/LightGBM what is next?

If I have feature importances of different variables in a xgboost/LightGBM model, how do I use this information? Is it better to just use the top n features and retrain the model? Does the feature ...
0
votes
1answer
33 views

Is it necessary to convert labels in string to integer for scikit_learn and xgboost?

I have a tabular data with labels that are in string. I will feed the data to decision trees in scikit_learn and XGBoost classifier. Is it necessary to convert the labels in string to integers for ...
0
votes
1answer
30 views

Is it necessary to normalise data for XGBoost?

MinMaxScaler in scikit_learn is used for data normalization (a.k.a feature scaling). Data normalisation is not necessary for decision trees. Since XGBoost is based on decision trees, is it necessary ...
1
vote
0answers
19 views

What is the best approach to train a multi-category regression model?

What is the best approach to train a multi-category regression model (using XBoost decision trees ensemble)? What are the ups and downs of each one? For example, if I want to train a model to predict ...
0
votes
1answer
23 views

Predicted and true values distributions comparison

Is this alarming when a distribution of predicted values differs from a distribution of true values? I use xgbregressor and get the following plots Usage of boxcox doesn't improve the case. My data ...
-1
votes
1answer
155 views

XGBoost vs ARIMA for Time Series analysis

Doing time series analysis, I have doubts on choosing the right model. I want to predict the next 30 mins window, from the input dataset which contains the no. of error count for that particular 1 min ...
0
votes
1answer
26 views

Predict items customers would buy in next order

I am working on a time series classification problem to identify what items customers would buy in their next order (customers orders different products every week). Let's say we have a customer who ...
1
vote
0answers
44 views

Trying to beat random forest with xgboost

I have a small time series dataset of about 3000 samples and 5 features. With xgboost, my predictions seem biased (consistently overestimating the target). No matter how many estimators I throw at the ...
0
votes
1answer
47 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my ...
0
votes
1answer
29 views

Understanding python XGBoost model dump output of a very simple tree

I am trying to understand the model dump output from XGBoost. I would like to step through and see exactly how the model arrived at it's prediction. To simplify I trained a model with 1 tree and 1 max ...
0
votes
1answer
19 views

Adding time as a feature with xgboost/random forests

I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
1
vote
0answers
57 views

Use LightGBM or FFM - imbalanced dataset

I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
0
votes
0answers
10 views

How to encode features that encode regular values as well as special categorical values

I was recently playing around with the FICO explainable machine learning challenge dataset. In the dataset, there are a bunch of numerical features which have values values typically in the 0-100 ...
0
votes
1answer
39 views

Specifying number of threads using XGBoost.train

When using the xgboost.train() function, all the threads are used. I would like to use a specific amount. Unfortunately, this function does not accept the ...
0
votes
1answer
62 views

xgboost feature selection and feature importance

when you create the new feature for data analysis for linear regression, it is clear that the feature has to be linear with other features is better but for xgboost what is the guideline to make a ...
1
vote
0answers
36 views

XGBoost predicting everything as null when sample weights are passed

I am trying to build an Uplift model using observational data. The data is consists of collections calls to customers and my objective is to predict the incremental probability due to the treatment (...
0
votes
1answer
42 views

XGBoost: # rounds is equal to n_estimators?

I'm running a regression XGBoost model and trying to prevent over-fitting by watching the train and test error using this code: ...
0
votes
1answer
26 views

How to find feature importance with multiple XGBoost models

My problem statement : Time Series forecasting(Month wise data), training on 96 months of data and predicting next 12 months with a 3 months empty window in between. Example : Batch 1 ...
8
votes
4answers
311 views

Why is there a difference between predicting on Validation set and Test set?

I have a XGBoost model trying to predict if a currency will go up or down next period (5 min). I have a dataset from 2004 to 2018. I split the data randomized into 95% train and 5% validation and the ...
2
votes
0answers
150 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
1
vote
1answer
51 views

Explaining XGBoost functioning to non-technical people

I have been tasked to explain the principle of the XGBoost algorithm to non-technical people (think 1-2 slides in a powerpoint presentation to upper management). I am currently working with the ...
4
votes
1answer
226 views

How to extract trees in XGBoost?

I want to extract each tree so that I can feed it with any data, and see the output. ...
0
votes
2answers
66 views

Compare scores of models

We got several models with predictions. How can we compare scores of different models with each other? We assume that we got xgboost models and scores distribution can be different for each model, so ...

1 2 3 4 5 … 7