Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1
vote
0answers
51 views

XGBoost predicting everything as null when sample weights are passed

I am trying to build an Uplift model using observational data. The data is consists of collections calls to customers and my objective is to predict the incremental probability due to the treatment (...
0
votes
1answer
53 views

XGBoost: # rounds is equal to n_estimators?

I'm running a regression XGBoost model and trying to prevent over-fitting by watching the train and test error using this code: ...
0
votes
1answer
28 views

How to find feature importance with multiple XGBoost models

My problem statement : Time Series forecasting(Month wise data), training on 96 months of data and predicting next 12 months with a 3 months empty window in between. Example : Batch 1 ...
8
votes
4answers
311 views

Why is there a difference between predicting on Validation set and Test set?

I have a XGBoost model trying to predict if a currency will go up or down next period (5 min). I have a dataset from 2004 to 2018. I split the data randomized into 95% train and 5% validation and the ...
2
votes
0answers
173 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
1
vote
1answer
51 views

Explaining XGBoost functioning to non-technical people

I have been tasked to explain the principle of the XGBoost algorithm to non-technical people (think 1-2 slides in a powerpoint presentation to upper management). I am currently working with the ...
4
votes
1answer
323 views

How to extract trees in XGBoost?

I want to extract each tree so that I can feed it with any data, and see the output. ...
0
votes
2answers
68 views

Compare scores of models

We got several models with predictions. How can we compare scores of different models with each other? We assume that we got xgboost models and scores distribution can be different for each model, so ...
0
votes
0answers
62 views

Save xboost model from R as Python pickle

I trying save R model to pickle, but I getting error Evaluation error: Unable to convert R object to Python type. ...
4
votes
2answers
647 views

L1 & L2 Regularization in Light GBM

This question pertains to L1 & L2 regularization parameters in Light GBM. As per official documentation: reg_alpha (float, optional (default=0.)) – L1 ...
6
votes
3answers
151 views

xgboost: Is there a way to perform regression on rates/percentages data?

I have a dependent variable, $Y$, that is made up of rates/percentages data, so each value is between $0$ and $1$. I was attracted to the xgboost library because it allows focusing in on specific ...
1
vote
1answer
178 views

When I use SHAP for classification problem, it shows an output that is not 0 or 1. How can I overcome this?

I'm using Pima Indians Diabetes Database(https://www.kaggle.com/uciml/pima-indians-diabetes-database). I made predictions using XGboost and I'm trying to analyze the features using SHAP. However ...
3
votes
2answers
83 views

Get feature importance for each observation with XGBoost

I have trained an XGBoost binary classifier and I would like to extract features importance for each observation I give to the model (I already have global features importance). More specifically, I ...
2
votes
2answers
80 views

unimportant features impact on model's performance

Using XGBoost and RandomForests, do unimportant features (according to the feature_importances_ attribute) hurt the model's performance? Do I need to carefully ...
0
votes
2answers
31 views

Xgboost take k best predictions

I have a mission of classification with a lot of classes. I am comparing some ML algorithms for this case and I would like to try xgboost. I am writing in python and I am trying to get the best 3 ...
1
vote
1answer
520 views

XGBOOST : model.predict_proba() and model.predict() conflicting behaviour

I have two classes : 1 and 2 The output of model.predict_proba() -> [0.333,0.6667] The output of model.predict() -> 1 This is happening for around 200 test values out of the test data of 10 lac. ...
0
votes
0answers
24 views

records with perfect correlation to the answer. Drop or Keep?

I have about 1000 records (5 numeric, 5 categorical vars) and about 25 of them have something in 5-level categorical variable that just gives away answer. It's just too obvious and I'm not worried ...
0
votes
0answers
73 views

After training an XGBoost model with a survival:cox objective, how can I get individual survival predictions for new data?

I have a large dataset and used the documentation found here to create an xgboost data object, trained my xgboost model, and then computed a c-statistic to determine the quality of the model. I am ...
1
vote
0answers
91 views

Improving recall in XGBoost algorithm

I have highly imbalanced dataset. I am using XGBoost and I got the following results without balancing the dataset out: Precision: 0.87 Recall: 0.79 F1: 0.83 My ...
4
votes
1answer
159 views

Interpretability of RMSE and R squared scores on cross validation

I'm working on a regression problem with 30k rows in my dataset, decided to use XGBoost mainly to avoid processing data for a quick primitive model. And i noticed upon doing cross-validation that ...
2
votes
2answers
113 views

Lowering learning rate makes my accuracy on the validation set go down

I'm using XGBoost and my mean absolute error on the validation set goes up when I change it from 0.05 to 0.03, I thought a smaller learning rate only makes it run slower and will if anything increase ...
5
votes
2answers
178 views

XGBoost, binary classification: uneven number of observations per user

I'm working on a binary classification problem with XGBoost and I have a dataset, which has uneven number of observations per user. For some users there are over 100 observations, whereas for some ...
1
vote
0answers
18 views

Is it possible to create nonbinary trees in XGBoost?

I'm looking through the documentation for XGBoost, and I'm not seeing any parameters relating to number of branches per node.
2
votes
2answers
727 views

Python XGBoost predict_proba returns very high or low probabilities

I trained my data with XGBoost in python with GridSearchCV as follows: ...
1
vote
0answers
30 views

I have tried 5 different types of model but all returns really low training accuracy (~64%) and low testing accuracy (~14%). What should I do?

I am working with a typical regressor problem. There are $6$ features in the dataset that I am concerned with. There are about $800$ data points in my dataset. The features and the predicted values ...
1
vote
2answers
38 views

Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
0
votes
0answers
33 views

How to test model accuracy on new vs. historical data?

I created an XG Boost model to predict churn using a dataset of customers who were sold during 2018. The accuracy of the model is 89%. Does it make more sense to re-pull the 2018 dataset, where more ...
1
vote
0answers
46 views

Feature Importance Scores from Gradient Boosting vs Random Forest

In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier. Would like to know what are the fundamental differences in how this attribute is ...
1
vote
1answer
90 views

Tweedie Loss for Keras

We are currently using XGBoost model with Tweedie loss for solving a regression problem which works very good, now I wanted to move our model to Keras and experience with neural networks, do anybody ...
0
votes
3answers
349 views

Xgboost Parameter Tuning

Are there methods to tune and train an xgboost model in an optimized time - when I tune paramaters and train the model it takes around 12 hours to execute? I would ...
2
votes
1answer
25 views

How can I get an algorithm to have an evalutation metric based on aggregate predictions?

Let's say I have a model that makes a prediction per individual. An example data set is below. Normally, evaluation metrics (for example within the XGBoost algorthim), are used at the individual ...
1
vote
0answers
35 views

How does the feval parameter influences the XGBoost training process?

In the package XGBoost, is possible to modify the feval (evaluated function) to a personalized one (as shown in the link: MAPE eval metric). I would like to know how is the training process of the ...
2
votes
1answer
70 views

Xgboost multiple class predictive performance beats one versus rest

I have an NLP task I'm tackling with xgboost (R implementation). Before describing my doubt I'll give you some background: I have a corpus of documents for which I did topic discovery, using a term ...
2
votes
1answer
79 views

Marketing Mix Model (MMM) using XGBoost?

Does anyone know of any literature on Marketing Mix Modelling (MMM) using XGBoost? Is this a viable techniques for MMM modelling? What would the advantages/disadvantages be? I think it would be a ...
0
votes
0answers
73 views

XGBClassifier Feature Order

I'm working with XGBClassifier from the XGBoost Python API and recently found out that the class requires feature columns to be in the same order for predictions as they were used for training. For ...
0
votes
3answers
52 views

How to interpret feature importance (XGBoost) in this case?

I found two dominant features from plot_importance. My dependent variable Y is customer retention (whether or not the customer will retain, 1=yes, 0=no). My ...
1
vote
2answers
33 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
0
votes
0answers
24 views

Using bagging and random forests together

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of ...
0
votes
1answer
14 views

Product Prediction to group of customers

I have multiple groups of customer, say for segment 1 as shown in the pictures, I have a list of products that I can choose the cross-sell to that group. Consider ...
1
vote
2answers
136 views

Xgboost in R: Dealing with extreme class imbalance

My data has an extreme class imbalance - 99.7% is 0's and 0.2% is 1's and almost all the predictor variables (6 out of 7) are categorical. I trained an xgboost classifier after performing an ...
4
votes
1answer
243 views

XGBoost Huge Dataset ~1TB

Can a gradient boosting solution like XGBoost or Lightbgm be used for a huge amount of data ? I have a csv file of 820GB containing 1 Billion observations and each observation has 650 datapoints. Is ...
0
votes
1answer
46 views

XGBoost: Can the features in test data be a subset of the features used to train the model?

Is it a problem if the test data only has a subset of the features that are used to train the xgboost model? All my predictor variables (except 1) are factors, so one hot encoding is done before ...
0
votes
1answer
23 views

How to interpret a random variable in the variable importance?

I have a problem, for simplicity let's say it is a binary classification problem. I am trying to solve this problem using XGBoost. A standard output plot for any ML algorithm, is the feature ...
0
votes
0answers
114 views

Bad input shape; XGB

I'm trying out a simple code to test the xgboost library in python. My input matrix has 17 features and 16,718 observations X = (16718,17) My output has 3 ...
1
vote
1answer
44 views

correct ML approach

I wanted to get your thoughts on a problem I have been facing. I have daily level product sales information (about 4 years). The sales are affected by the typical factors such as seasonality, day of ...
0
votes
0answers
69 views

XGBoost not learning

I have developed a train set for XGBoost to apply a learning to rank function on top of with the following parameters: ...
2
votes
1answer
309 views

Is the number of iterations in gradient tree boosting just the number of trees?

I have been searching for a while and I just can't find any indication. When people talk about iterations in algorithms like XGBoost or LightGBM, or Catboost, do they mean how many decision trees i.e. ...
0
votes
1answer
54 views

GBM: small change in the trainset causes radical change in predictions

I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
0
votes
3answers
69 views

Can this problem be solved using deep learning?

I want to predict price of used cars. I have data like this: Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.? I used one hot approach for nominal features and ...
0
votes
1answer
1k views

What is out come cox regression in xgboost

I am running xgboost where objective is survival:cox and eval_metric is cox-nloglik. Y range from -800 to 800. However, predicted values are way to large in range from 10^3 to 10^13. I am not sure why ...