Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

163 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
733 views

Feeding data to Xgboost for recomender system

I am using xgboost for a recommender system. There are 3-4 recommended content on each page. My data consists of columns like page_id and advertisement_id. Currently for every page_id, there are 3-4 ...
1
vote
0answers
299 views

What does “gblinear” do in XGBoost?

For example, if I run a single round (nrounds=1), how does XGBoost go about making predictions? I thought it would simply return a linear regression model, but I ...
1
vote
2answers
512 views

Xgboost in R: Dealing with extreme class imbalance

My data has an extreme class imbalance - 99.7% is 0's and 0.2% is 1's and almost all the predictor variables (6 out of 7) are categorical. I trained an xgboost classifier after performing an ...
1
vote
1answer
431 views

XGBOOST missing_value feature degrades my performance?

I'm training an xgboost model for gout disease on a training set I sampled 1-to-7 case-control ratio (enriched in cases). I have 220 features and I reach a cross-...
0
votes
2answers
25 views

Using Transaction Amount to Guide Learning in an Fraud Detection Machine Learning Model

I am currently using transaction amount as a feature in an XGBoost classification model designed to identify fraudulent transactions. Furthermore, transaction amount is bounded for this problem ...
0
votes
0answers
14 views

kmeans cluster results for use as xgboost feature

I am curious if kmeans clusters can be used as xgboost features, along with the original features? Specifically for features X and labels Y can we: Split into X_train, y_train & X_test, y_test ...
0
votes
0answers
19 views

XGBoost: Typical gamma and min_child_weight range

What is the typical accepted range of gamma and min_child_weight parameters for the XGBoost algorithm? Is the range of min_child_weight correlated with the number of feature or samples in the training ...
0
votes
0answers
27 views

Confusion Matrix after XGBoost is showing positive as negative class

Please can you help me with confusion matrix. I've implemented the XGBoostClassifier. After fitting the model when I looked to the confusion matrix to view the performance on Test data. The confusion ...
0
votes
0answers
32 views

XGBoost Regressor model reproducibility

I am running an XGBoost Regressor to predict electricity consumption (load) and further classify predicted values as peaks or not. As for dataset I started with hourly energy load data + hourly ...
0
votes
0answers
17 views

Is there a python package that includes decision tree structures that can be used with a genetic algorithm?

I'd like to use decision tree / forest but I need to use a special objective function that can't be differentiated and hence I can't use XGBOost etc. That leaves a genetic algorithm where I use the ...
0
votes
0answers
14 views

sklearn GradientBoostingClassifier provides unstable predictions

I am using sklearn.ensemble.GradientBoostingClassifier to build a rather simple model which predicts probabilities. I have simplified the problem to having only one numerical predictor and the outcome ...
0
votes
0answers
21 views

how to represent feature importance in xgboost in percentage?

I am looking for a way to represent the feature importance numbers in percentage. I read through articles and API documentation for XGboost in python and it gives me the feature importance score, ...
0
votes
0answers
9 views

How should I add reference key column to output after modeling is Complete?

0 I have created a csv of data with the following columns: (1) app_key (2) churn, (3)tenure https://i.stack.imgur.com/NAlFF.png I have performed the following code in order to drop app_key and churn <...
0
votes
0answers
31 views

Probabilities predicted by XGBoost

I looking at football data and trying to predict whether a goal will occur using xgboost with objective binary: logistic. My data is 1:10 unbalanced with no goals being more dominant. I have used ...
0
votes
0answers
28 views

Feature Selection before modeling with Boosting Trees

I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
0
votes
0answers
66 views

CNN with XGBoost as an output layer, is it better?

Context: I have been experiencing with some Kaggle datasets to learn more about image classification. So, in this binary image classification task, I tried something that I thought would increase the ...
0
votes
0answers
22 views

How much does each tree of gradient boosting contribute to the global feature importance?

Let's say we are training a GBDT in the Titanic dataset. We have 3 trees in the GBDT. You extract the first tree and calculate the feature importance (no matter if cover, gain...), and Age importance =...
0
votes
0answers
67 views

Understanding interplay between eval_metric, metric and first_metric_only parameters in LGBMClassifier

In python API of LGBMClassifier, the constructor takes parameters metric and first_metric_only. Their descriptions are as ...
0
votes
0answers
31 views

what exactly is the hessian matrix in xgboost?

i would like to understand a bit more the mathematics behind xgboost. I understand that the hessian is the second partial derivative of the loss function, i originally thought this was with respect to ...
0
votes
0answers
22 views

Why do you need to update the number of boosting rounds each time you update a parameter in xgboost?

I have been reading material that suggesting that, after each grid search you do on a single parameter (e.g say on learning rate), you should update the number of boosting rounds afterwards. This is ...
0
votes
0answers
18 views

if gbm overfit, will variable importance calculation be accurate

If we are building GBM and we have too many trees that leads to overfitting, will the variable importance be affected?
0
votes
1answer
107 views

Correct theoretical regularized objective function for XGB/LGBM (regression task)

I am writing an academic paper on the application of Machine Learning methods to Time Series Forecasting and I am unsure about how to write down the theoretical part about the regularized objective ...
0
votes
0answers
54 views

What are references for problems where XGboost was successfully used?

I created a successful XGboost model but my boss would like me to give him references on the xgboost, where it was successfully used. Do you happen to know if any company used it for any purpose? ...
0
votes
0answers
55 views

What is the best way to train a gradient boosted model on a binomial dataset where the number of “observations” for each instance varies?

I have been trying to figure out the best way to train a gradient boosted model on a binomial dataset. To be more clear my dataset is in a format similar to this: [link to toy dataset]. https://i....
0
votes
0answers
16 views

using logistic:binary as objective in xgboost

i am using 'binary:logisitic' as my objective function in xgboost classifier. Therefore when i predict probabilities i.e: model.predict_proba(x_test)[:, 1] will ...
0
votes
0answers
36 views

how can i plot probability distribution of my classes in the way below?

All, I would like to plot the following: I have a binary classification problem where I am using xgboost as my 'model' below: ...
0
votes
0answers
79 views

Confidence score over xgboost logistic regression

The probabilities of logistic regression indicate how the certain the model is over predictions. if its 0.93 it means the model is 93% confident the label is 1 and 7% to be 0. or if the probability is ...
0
votes
0answers
44 views

XGBoost model performance barely budging, and max_depth is basically irrelevant

I have a dataset that includes 62 features and around 1 million observations. The 62 features are mostly socioeconomic status indicators for students as they start a school year. The labels are the ...
0
votes
0answers
14 views

XGBoost parameters that will overweight errors for a specific decile

My goal is to make predictions about the bottom decile of my dataset. If I can predict the bottom decile values well (ie. minimize the errors there) then I am not worried about making accurate ...
0
votes
1answer
257 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables ...
0
votes
0answers
34 views

XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?

We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
0
votes
0answers
15 views

Building Uplift Tree using boosting

I want to build an Uplift Model for multiple treatments. To get a good model, I would like to use boosting. How is it possible to use boosting with uplift modeling although we can't really calculate ...
0
votes
0answers
13 views

How Xgboost or Decision trees deal with discontinous data in tabular form for time series

I am participating in a kaggle competition by the name "predict future sales". There the objective is to predict sales for each shop for each item for next month. The data is given such that only ...
0
votes
0answers
29 views

Gamma objective function XGBoost

I am using XGBoost to predict a variable that is highly skewed and always is greater than zero. I did a significant search to see some materials for gamma objective function in XGBoost but I could not ...
0
votes
0answers
19 views

News Recommendation Engine with XGBoost

I want to build a news recommendation engine with XGBoost, but the data I have contains implicit user ratings, view history of a user. I know what my X's will(user embeddings + Item Embeddings) be but ...
0
votes
0answers
36 views

Sagemaker - XGBOOST rank:ndcg

Does anyone know if the rank:ndcg is available on AWS Sagemaker? I am currently trying to run a model, but it seems like it's not implemented. Am I using an older xgboost version? Kinda new to AWS ...
0
votes
0answers
26 views

Predict best score on unlabelled test set

Data I have one dataset with $1500$ data points, each with $\sim 23 000$ features (gene expression data, if that matters). However, I've split this dataset into a labelled training set of size 1000, ...
0
votes
0answers
11 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
0
votes
0answers
90 views

ExtraTreeClassifier does not handle missing values

I am using sklearn.tree.ExtraTreeClassifier. It does not handle missing value in training data. All tree-based algorithms handle missing value internally. So, is ...
0
votes
1answer
258 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
0
votes
0answers
88 views

XGBoost Python API - performance by API, R vs Python inconsistencies, GPU memory and verbosity

I'm combining a few questions together as I feel that it could benefit others. XGBoost API Is there a performance boost (training time or accuracy) when you use the learning API vs the Scikit-Learn ...
0
votes
0answers
30 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
0
votes
0answers
32 views

Multi-label classification yielding too much unlabeled rows

I am performing multi-label classification with xgboost + OneVsRestClassifier from sklearn. ...
0
votes
0answers
15 views

Getting bad predictions for high true values of target variable

I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
0answers
400 views

SHAP Kernel explainer for my pipeline model

I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer. The data contains different types of ...
0
votes
0answers
43 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
0
votes
0answers
89 views

Prediction error without having a true value

Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
0
votes
0answers
661 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
0
votes
1answer
136 views

Predict items customers would buy in next order

I am working on a time series classification problem to identify what items customers would buy in their next order (customers orders different products every week). Let's say we have a customer who ...
0
votes
1answer
716 views

Specifying number of threads using XGBoost.train

When using the xgboost.train() function, all the threads are used. I would like to use a specific amount. Unfortunately, this function does not accept the ...