Data Science

Questions tagged [xgboost]

Ask Question

For questions related to the eXtreme Gradient Boosting algorithm.

354 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Multi-output classifier using Tensorflow/Keras on tabular data

I have been tasked with combining a several classifier models we have into one model using deep learning (or something else). The reason for this is that, in future, it would be difficult to maintain ...
0
votes
0answers
10 views

Changing objective functions with penalty term in XGboost in R

I basically have something like: ...
0
votes
0answers
15 views

Examples of the use of xgboost for recommender systems?

Are there any state-of-the-art implementations of xgboost in recommender systems? I'm looking for GitHub implementations but also papers that discuss this. I've only found this paper https://...
1
vote
0answers
13 views

weighted quantile sketch in xgboost

I am unable to understand what is weighted quantile sketch in xgboost. Can anyone help me give an intuitive understanding of this?
1
vote
0answers
32 views

LIME is observing categorical features even though I am not passing any categorical features

Here is the code: ...
1
vote
1answer
14 views

Lime Explainer: ValueError: training data did not have the following fields

I'm attempting to gather ID level drivers from my XGBoost classification model using LIME and I'm running into some odd errors. I'm using this link as a reference. Here is the overall code that I'm ...
0
votes
0answers
24 views

Unable to import xgboost 0.9

When running import xgboost as xgb print(xgb.__version__) I am getting the following error: ...
4
votes
1answer
49 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to ...
4
votes
1answer
60 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
48 views

Prediction error without having a true value

Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
1
vote
1answer
18 views

best way to regularize gradient boosting regressor?

i am testing gradient boosting regressor from sklearn for time series prediction on noisy data (currency markets). https://scikit-learn.org/stable/modules/generated/sklearn.ensemble....
0
votes
1answer
13 views

sklearn.feature_selection vs xgboost feature_importances?

sklearn.feature_selection vs xgboost feature_importances Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
0
votes
0answers
20 views

Why XGBoost regressor predicts behavior but not the amplitude?

I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling). I found out that it works very well for predicting the ...
1
vote
1answer
32 views

Average of importance gain for a categorical variable

Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc). I train an XGBoost model on a numeric ...
1
vote
1answer
25 views

XGBClassifier: make the output of predict_proba ascending regarding a specific feature

I'm trying to build a classifier using Xgboost on some high dimensional data, the problem I'm having is that I have the prior knowledge that the output probabilities should be ascending regarding a ...
0
votes
0answers
18 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
1
vote
0answers
16 views

SHAP Explanations in case of repeated train/test split

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that ...
1
vote
2answers
28 views

Adding extra variables to XGboost model is worsening the train and test accuracy

I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split. However, when I am adding two new features , the ...
1
vote
0answers
23 views

Feature Vectors representation

I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
0
votes
0answers
55 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
1
vote
0answers
43 views

Hyperparameter Tuning using Bayesian Techniques

I've been looking into Bayesian optimization for hyperparameter tuning and trying to compare the results I get to those I get using different methods (random grid search). I came across this site, ...
2
votes
0answers
28 views

Can I specify the root node splitting feature in XGBoost?

Just what the title says. Suppose I know the feature that I want to be used for splitting for the root node in a tree model in XGBoost; is there a way for me to tell XGBoost to split on this feature ...
0
votes
0answers
19 views

Feature engineering ideas with dates, coordinates and other variables

I'm working on an ETA problem where I'm trying to estimate a time of arrival for a delivery. I have coordinates of pickup/destination, time of pick , infos about the rider, some other variables that ...
0
votes
0answers
68 views

Suggestion for handling specific missing data

I have data, that describes distance from given location to nearest object (e.g. school, shop etc). Because of performance reasons I couldn't scrape the data about objects, that are futher away than 2....
0
votes
0answers
11 views

Improve the results of imbalanced multi-classification multi-lables data

I have 10k rows of multi-classification (x1..x27,y), size of dataframe is: 28*10k and its ...
4
votes
2answers
83 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...
1
vote
1answer
52 views

How does XGBoost use softmax as an objective function?

I'm quite used to seeing functions like log-loss, RMSE, cross entropy as objective functions and it's easy to imagine why minimizing these would give us the best model. What's difficult to imagine is ...
0
votes
2answers
74 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
0
votes
2answers
118 views

What is the difference between gradient descent and gradient boosting? Are they interdependent on each other by any way?

What is the difference between gradient descent and gradient boosting? Are they interdependent on each other in any way ?
0
votes
1answer
41 views

XGBoost probability distribution tending towards the extreme

I am using an XGBoost classifier to make risk predictions, and I see that even if it has very good binary classification results, the probability outputs are mainly under $0.05$ or over $0.95$ (like ...
0
votes
1answer
26 views

Similarity of XGBoost models?

Is xgboost with n_estimators = 100 and learning_rate = 0.1, same as xgboost with n_estimators = 50 and learning_rate = 0.2 ?
1
vote
0answers
26 views

XGBoost speed issues

I'm trying to optimize the hyperparameters for XGBoost, thus needing to run it multiple times with different parameters. However the time needed to run single XGBoost with the parameters provided ...
0
votes
0answers
16 views

Data Transformation Tips for xgboost's XGBClassifier

I have this X_train and test distribution for the 4 features 'X', 'Y', 'TX' and 'TY'. I realize the range of the distribution is widely varying .. Can you suggest a good way to clean/ transform that ...
3
votes
2answers
160 views

XGBoost validation for number of trees

I have a simple Question: I am using XGBoost to classify some data: 1.) With 100 estimators I have following scores(roc_score): train_data : 98.5 validation_data : 97.2 2.) With 500 ...
0
votes
1answer
19 views

How to tell a boosting model that 2 features are related and should not be interpreted stand-alone?

I am using XGBoost for a machine learning model that learns from tabular data. XGBoost uses boosting method on decision trees. When I look at the decision-making logic of decision trees, I notice the ...
1
vote
1answer
25 views

xgboost in R have different results compared to boosted decision tree in Azure ML

I have a small data set (4000 records with 10 features) and I used XGBOOST in R as well as Boosted Decision Tree model in Azure ML studio. Unfortunately the results are different. I like to optimize ...
4
votes
1answer
147 views

Forecasting via LSTM or XGBoost… is it really a forecast or

I guess I understand the idea of predictions made via LSTM or XGBoost models, but want to reach out to the community to confirm my thoughts. This tutorial does a nice job explaining step by step of ...
0
votes
0answers
27 views

if I got feature importance of xgboost/LightGBM what is next?

If I have feature importances of different variables in a xgboost/LightGBM model, how do I use this information? Is it better to just use the top n features and retrain the model? Does the feature ...
0
votes
1answer
61 views

Is it necessary to convert labels in string to integer for scikit_learn and xgboost?

I have a tabular data with labels that are in string. I will feed the data to decision trees in scikit_learn and XGBoost classifier. Is it necessary to convert the labels in string to integers for ...
0
votes
1answer
47 views

Is it necessary to normalise data for XGBoost?

MinMaxScaler in scikit_learn is used for data normalization (a.k.a feature scaling). Data normalisation is not necessary for decision trees. Since XGBoost is based on decision trees, is it necessary ...
1
vote
0answers
19 views

What is the best approach to train a multi-category regression model?

What is the best approach to train a multi-category regression model (using XBoost decision trees ensemble)? What are the ups and downs of each one? For example, if I want to train a model to predict ...
0
votes
1answer
24 views

Predicted and true values distributions comparison

Is this alarming when a distribution of predicted values differs from a distribution of true values? I use xgbregressor and get the following plots Usage of boxcox doesn't improve the case. My data ...
-1
votes
1answer
258 views

XGBoost vs ARIMA for Time Series analysis

Doing time series analysis, I have doubts on choosing the right model. I want to predict the next 30 mins window, from the input dataset which contains the no. of error count for that particular 1 min ...
0
votes
1answer
26 views

Predict items customers would buy in next order

I am working on a time series classification problem to identify what items customers would buy in their next order (customers orders different products every week). Let's say we have a customer who ...
1
vote
0answers
53 views

Trying to beat random forest with xgboost

I have a small time series dataset of about 3000 samples and 5 features. With xgboost, my predictions seem biased (consistently overestimating the target). No matter how many estimators I throw at the ...
0
votes
1answer
47 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my ...
0
votes
1answer
44 views

Understanding python XGBoost model dump output of a very simple tree

I am trying to understand the model dump output from XGBoost. I would like to step through and see exactly how the model arrived at it's prediction. To simplify I trained a model with 1 tree and 1 max ...
0
votes
1answer
19 views

Adding time as a feature with xgboost/random forests

I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
1
vote
0answers
80 views

Use LightGBM or FFM - imbalanced dataset

I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
0
votes
0answers
10 views

How to encode features that encode regular values as well as special categorical values

I was recently playing around with the FICO explainable machine learning challenge dataset. In the dataset, there are a bunch of numerical features which have values values typically in the 0-100 ...

1 2 3 4 5 … 8