Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1
vote
2answers
28 views

Adding extra variables to XGboost model is worsening the train and test accuracy

I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split. However, when I am adding two new features , the ...
1
vote
1answer
141 views

Can you reuse observations from your train data in your final test data?

For an employee population, I am trying to determine who among the employees are likely to get injured in the future based on 2 years worth of data. Unlike in most machine learning problems where you ...
2
votes
2answers
3k views

difference between XGBRegressor and XGBClassifier

I'm trying to understand the difference between xgboost.XGBRegressor and xgboost.sklearn.XGBClassifier. Can someone explain the difference in a concise manner? Because when I fit both classifiers ...
1
vote
1answer
91 views

Tweedie Loss for Keras

We are currently using XGBoost model with Tweedie loss for solving a regression problem which works very good, now I wanted to move our model to Keras and experience with neural networks, do anybody ...
0
votes
0answers
4 views

Examples of the use of xgboost for recommender systems?

Are there any state-of-the-art implementations of xgboost in recommender systems? I'm looking for GitHub implementations but also papers that discuss this. I've only found this paper https://...
1
vote
0answers
12 views

weighted quantile sketch in xgboost

I am unable to understand what is weighted quantile sketch in xgboost. Can anyone help me give an intuitive understanding of this?
0
votes
1answer
2k views

How to show progress of sklearn.multioutput.MultiOutputRegressor and XGBRegressor?

Is it possible to show the training progress of the MultiOutputRegressor in sklearn? When a huge dataset is processed, my program runs a long time and I have no clue how long it will take. I have ...
1
vote
1answer
18 views

best way to regularize gradient boosting regressor?

i am testing gradient boosting regressor from sklearn for time series prediction on noisy data (currency markets). https://scikit-learn.org/stable/modules/generated/sklearn.ensemble....
0
votes
1answer
80 views

XGBOOST CV() producing error

I am getting the following error while using xgboost.cv() (scikit-learn interface). I am working on a regression problem. Below is the code and trace. No idea why ...
1
vote
1answer
117 views

XGBOOST missing_value feature degrades my performance?

I'm training an xgboost model for gout disease on a training set I sampled 1-to-7 case-control ratio (enriched in cases). I have 220 features and I reach a cross-...
0
votes
2answers
37 views

How to ensure same encoding pattern?

I created a XGBRegressor model with certain encoded 'object' dtypes in the data. Now if I want to run the model with new set of data which is freshly encoded it's giving wrong predictions. How to ...
1
vote
1answer
12 views

Lime Explainer: ValueError: training data did not have the following fields

I'm attempting to gather ID level drivers from my XGBoost classification model using LIME and I'm running into some odd errors. I'm using this link as a reference. Here is the overall code that I'm ...
0
votes
0answers
21 views

Unable to import xgboost 0.9

When running import xgboost as xgb print(xgb.__version__) I am getting the following error: ...
6
votes
1answer
47 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to ...
4
votes
1answer
57 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
47 views

Prediction error without having a true value

Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
1
vote
1answer
32 views

Average of importance gain for a categorical variable

Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc). I train an XGBoost model on a numeric ...
1
vote
2answers
33 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
1
vote
1answer
430 views

Using a combination of gradient boosting with LSTM for classification?

I am presently using an LSTM model to classify high dimensional tabular data which is not text/images (dimensions 21392x1970). I also tried XGBoost (Gradient boosting) in Python separately for the ...
1
vote
2answers
137 views

Xgboost in R: Dealing with extreme class imbalance

My data has an extreme class imbalance - 99.7% is 0's and 0.2% is 1's and almost all the predictor variables (6 out of 7) are categorical. I trained an xgboost classifier after performing an ...
0
votes
3answers
69 views

Can this problem be solved using deep learning?

I want to predict price of used cars. I have data like this: Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.? I used one hot approach for nominal features and ...
0
votes
1answer
15 views

Product Prediction to group of customers

I have multiple groups of customer, say for segment 1 as shown in the pictures, I have a list of products that I can choose the cross-sell to that group. Consider ...
0
votes
1answer
749 views

XGBOOST (sklearn interface) REGRESSION error

I am trying to run a GRIDSEARCHCV (sklearn) on XGBRegressor. Documentation on the parameter says that if regression, then objective = reg:squarederror.(see https://...
1
vote
1answer
13 views

sklearn.feature_selection vs xgboost feature_importances?

sklearn.feature_selection vs xgboost feature_importances Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
6
votes
3answers
152 views

xgboost: Is there a way to perform regression on rates/percentages data?

I have a dependent variable, $Y$, that is made up of rates/percentages data, so each value is between $0$ and $1$. I was attracted to the xgboost library because it allows focusing in on specific ...
11
votes
3answers
560 views

XGBoost outputs tend towards the extremes

I am currently using XGBoost for risk prediction, it seems to be doing a good job in the binary classification department but the probability outputs are way off, i.e., changing the value of a feature ...
0
votes
0answers
20 views

Why XGBoost regressor predicts behavior but not the amplitude?

I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling). I found out that it works very well for predicting the ...
1
vote
1answer
25 views

XGBClassifier: make the output of predict_proba ascending regarding a specific feature

I'm trying to build a classifier using Xgboost on some high dimensional data, the problem I'm having is that I have the prior knowledge that the output probabilities should be ascending regarding a ...
0
votes
0answers
17 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
1
vote
0answers
15 views

SHAP Explanations in case of repeated train/test split

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that ...
0
votes
0answers
48 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
1
vote
0answers
22 views

Feature Vectors representation

I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
3
votes
1answer
649 views

Imbalanced dataset with 3 classes xgboost scale_pos_weight parameter

The xgboost classifier states the use of parameter scale_pos_weight for 2-class problems. I have a highly imbalanced dataset with 3 classes. Classes '1' and '-1' ...
0
votes
1answer
26 views

Predict items customers would buy in next order

I am working on a time series classification problem to identify what items customers would buy in their next order (customers orders different products every week). Let's say we have a customer who ...
0
votes
0answers
60 views

Making a model to predict the error of another model

So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to ...
1
vote
1answer
25 views

xgboost in R have different results compared to boosted decision tree in Azure ML

I have a small data set (4000 records with 10 features) and I used XGBOOST in R as well as Boosted Decision Tree model in Azure ML studio. Unfortunately the results are different. I like to optimize ...
2
votes
2answers
595 views

Ordinal classification with xgboost

I am working in the problem where the dependent variables are ordered classes, such as bad, good, ...
1
vote
1answer
44 views

correct ML approach

I wanted to get your thoughts on a problem I have been facing. I have daily level product sales information (about 4 years). The sales are affected by the typical factors such as seasonality, day of ...
1
vote
0answers
43 views

Hyperparameter Tuning using Bayesian Techniques

I've been looking into Bayesian optimization for hyperparameter tuning and trying to compare the results I get to those I get using different methods (random grid search). I came across this site, ...
0
votes
0answers
68 views

Suggestion for handling specific missing data

I have data, that describes distance from given location to nearest object (e.g. school, shop etc). Because of performance reasons I couldn't scrape the data about objects, that are futher away than 2....
0
votes
1answer
54 views

GBM: small change in the trainset causes radical change in predictions

I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
2
votes
1answer
562 views

What does xgb's scale_pos_weight parameter do for regression?

From other posts (see Unbalanced multiclass data with XGBoost) and the documentation, scale_pos_weight in XGBoost appears to balance positive and negative cases, ...
3
votes
1answer
575 views

Friedman H'statistic for interaction

I'm interested in interaction effects in Random Forest. I try to implement Friedman H'statistic basically developped for Gradient boosting Friendman, 2001. But I have no idea how calculate (44) on ...
2
votes
0answers
28 views

Can I specify the root node splitting feature in XGBoost?

Just what the title says. Suppose I know the feature that I want to be used for splitting for the root node in a tree model in XGBoost; is there a way for me to tell XGBoost to split on this feature ...
0
votes
0answers
19 views

Feature engineering ideas with dates, coordinates and other variables

I'm working on an ETA problem where I'm trying to estimate a time of arrival for a delivery. I have coordinates of pickup/destination, time of pick , infos about the rider, some other variables that ...
0
votes
0answers
11 views

Improve the results of imbalanced multi-classification multi-lables data

I have 10k rows of multi-classification (x1..x27,y), size of dataframe is: 28*10k and its ...
0
votes
1answer
47 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my ...
0
votes
1answer
570 views

On which algorithm for boosting is the method xgbLinear of the xgboost/caret- package based on?

In caret package of R, there is a method 'xgblinear'. What is the working algorithm behind this method.
4
votes
2answers
81 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...