Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
39 views

xgboost in R have different results compared to boosted decision tree in Azure ML

I have a small data set (4000 records with 10 features) and I used XGBOOST in R as well as Boosted Decision Tree model in Azure ML studio. Unfortunately the results are different. I like to optimize ...
4
votes
2answers
111 views

XGBOOST - different result between train_test_split and manually splitting

I am trying to train XGBOOST model. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y) when I'm using ...
1
vote
1answer
23 views

Interpretable xgboost - Calculate cover feature importance

When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance. There are same parameters in the xgb api sucha as: weight, gain, cover, ...
3
votes
1answer
32 views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
4
votes
2answers
99 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
5
votes
1answer
87 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
-1
votes
1answer
18 views

Machine Learning Out of test data forecast (XGBoost, ANN)

I see a lot of applications for machine learning techniques applied to time series. Unfortunately almost all kernels with XGBoost or ANN stop short in creating an actual forecast. The achieve a great ...
0
votes
0answers
25 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
2
votes
1answer
24 views

Each time the result of XGBoost is different

I'm running xgboost and I fix the seed number but the result is different for each time when I rerun the xgboost with same dataset ...
1
vote
1answer
60 views

GBM: small change in the trainset causes radical change in predictions

I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
2
votes
1answer
602 views

What does xgb's scale_pos_weight parameter do for regression?

From other posts (see Unbalanced multiclass data with XGBoost) and the documentation, scale_pos_weight in XGBoost appears to balance positive and negative cases, ...
1
vote
0answers
17 views

xgboost result is not stable [closed]

Here is my xgboost hyperparameters ...
-1
votes
1answer
21 views

Comparing Classification Algorithms [closed]

I have found the below code. My question is simple as I am an aspiring Data Scientist. can I rely on the output of this code spinet to choose the algorithms that i would use to model a problem? This ...
-1
votes
0answers
10 views

How to get f_score from xgbRegressor and what is the difference between plot_importance and feature_importance

I am a bit new to python, trying to use xgbRegressor for "feature selection". In order to extract the score of each feature from this process, by using this code below "x = list(zip(X_set,...
2
votes
1answer
140 views

XGBoost predicting everything as null when sample weights are passed

I am trying to build an Uplift model using observational data. The data is consists of collections calls to customers and my objective is to predict the incremental probability due to the treatment (...
1
vote
0answers
15 views

Multi-label classification yielding too much unlabeled rows

I am performing multi-label classification with xgboost + OneVsRestClassifier from sklearn. ...
4
votes
2answers
3k views

changing cost function in xgboost

I'm using the newest version of xgboost package in python 2.7 and based on my problem, I'm going to change xgboost cost function to use my own defined cost function. Couple of questions: In which ...
1
vote
1answer
17 views

feature importance and xgboost?

Let say I got feature importance for xgclassifier ...
1
vote
0answers
18 views
0
votes
1answer
50 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my ...
0
votes
1answer
9 views

is there cross validation for xgb classification for multi labels?

is there cross validation for xgb classification for multi labels? I have been search but can not find any cross validation for xgb classifier is using cross validation for xgb or xgb classifier ...
0
votes
1answer
615 views

On which algorithm for boosting is the method xgbLinear of the xgboost/caret- package based on?

In caret package of R, there is a method 'xgblinear'. What is the working algorithm behind this method.
14
votes
4answers
669 views

XGBoost outputs tend towards the extremes

I am currently using XGBoost for risk prediction, it seems to be doing a good job in the binary classification department but the probability outputs are way off, i.e., changing the value of a feature ...
1
vote
1answer
17 views

is there any rule to apply pca to the imbalance data? [closed]

Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost) I used multiclass imbalance data to pca but the log-loss accuracy getting decrease any theoritical background of this?
0
votes
0answers
10 views

Getting bad predictions for high true values of target variable

I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
2
votes
2answers
89 views

Compare scores of models

We got several models with predictions. How can we compare scores of different models with each other? We assume that we got xgboost models and scores distribution can be different for each model, so ...
1
vote
2answers
32 views

Can an entire data frame be used as a prediction variable?

I am attempting to use XGBoost in R to train a model that predicts a fixed number of target variables using all data from previous dates, as well as the two categorical variables (...
0
votes
1answer
24 views

XGBoost: How to set the probability threshold for multi class classification

I am using the XGBoost for classification of text data. There are the 3 different classes in the training dataset. ...
1
vote
1answer
26 views

order of features for model tuning vs model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
0
votes
1answer
14 views

order of features importance after make_column_transformer and pipeline

I have a data preparation and model fitting pipeline that takes a dataframe (X_trn) and uses the ‘make_column_transformer’ and ‘Pipeline’ functions in sklearn to prepare the data and fit XGBRegressor. ...
4
votes
1answer
139 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
5
votes
1answer
593 views

Custom objective function in xgboost for Regression

I am working on a regression problem, where I want to modify the loss function in xgboost library such that my predictions should never be lesser than the actual value. I have written this code: <...
4
votes
1answer
160 views

How are samples selected from training data in Xgboost

In Random Forest, each tree is not fed with the full batch of training data, only a sample. How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
1
vote
1answer
23 views

How to check for “statistical significance” of categorical feature in black box models

Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
1
vote
1answer
23 views

How to calculate joint feature contribution for XGBoost Classifier in python?

I referred to http://savvastjortjoglou.com/intrepretable-machine-learning-nfl-combine.html#Joint-Feature-Contributions this beautiful document to research about joint feature contibutions. But this ...
0
votes
1answer
151 views

Specifying number of threads using XGBoost.train

When using the xgboost.train() function, all the threads are used. I would like to use a specific amount. Unfortunately, this function does not accept the ...
-1
votes
0answers
55 views

What are the pros and cons of XGBoost?

I just found some pros and cons for tree methods in general (https://medium.com/syncedreview/tree-boosting-with-xgboost-why-does-xgboost-win-every-machine-learning-competition-ca8034c0b283). What are ...
0
votes
1answer
67 views

DataFrame.dtypes error when training model using xgboost

I am getting a DataFrame.dtypes error while following the last steps of this tutorial: https://www.hackerearth.com/practice/machine-learning/machine-learning-projects/python-project/tutorial/. Here ...
0
votes
0answers
51 views

SHAP Kernel explainer for my pipeline model

I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer. The data contains different types of ...
2
votes
0answers
26 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python? There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv? ...
2
votes
1answer
55 views

Training a model where each response in the observation data has a different known varience

I have a dataset where each response variable is the number of successes of N Bernoulli trials with N and p (the probability of success) being different for each observation. The goal is to train a ...
3
votes
1answer
73 views

Explaining XGBoost functioning to non-technical people

I have been tasked to explain the principle of the XGBoost algorithm to non-technical people (think 1-2 slides in a powerpoint presentation to upper management). I am currently working with the ...
1
vote
0answers
26 views

is it possible (and/or logical) to set feature importance for xgboost?

If I understand tree based methods correctly, it would be better for more important features to be toward the top of the tree. Is there a way I can dictate this in xgboost? Similar to how I can ...
2
votes
2answers
964 views

Minimum number of samples to train XGBoost without overfitting

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron. Is there a similar rule of thumb for XGBoost, ...
1
vote
1answer
1k views

XGBOOST : model.predict_proba() and model.predict() conflicting behaviour

I have two classes : 1 and 2 The output of model.predict_proba() -> [0.333,0.6667] The output of model.predict() -> 1 This is happening for around 200 test values out of the test data of 10 lac. ...
1
vote
3answers
30 views

Categorical variables with multiple entries transformed to entity embedding

I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction. Some ...
1
vote
1answer
238 views

correct setting of eval_set in multiclass classification xgboost python , error is “ Check failed: preds.size() == info.labels_.size()”

i have a multiclass classification problem with 3 classes [-1,0,1] . i'd like to use eval_set in xgboost. but it fails with error: ...
1
vote
3answers
583 views

Xgboost Parameter Tuning

Are there methods to tune and train an xgboost model in an optimized time - when I tune paramaters and train the model it takes around 12 hours to execute? I would ...
3
votes
1answer
94 views

How the algorithms such as Xgboost and LightGBM fills the NAN (missing data)?

How the algorithms such as Xgboost and LightGBM fills the NAN ? I know that they learn how to fill it but how they do it is my question and what is they method of learning how to fill the NAN?
4
votes
1answer
124 views

xgboost or lightgbm to handle Binomial problems

I have a dataset containing a column of trials, a column of successes and other features; and, obviously, I can generate a probability column. I would like to use gradient boosting methods (like ...

1 2 3 4 5 8