Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
405
questions
0
votes
1answer
17 views
Can you combine two xgboost models into one?
If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
1
vote
0answers
24 views
State of the Art/Research 2020 of Time Series Forecasting/Prediction
Im looking for the state of the art/research of time series data for forcasting/prediction.
As far as im aware it is Extrem Gradient Boosting (XGBoost) or LSTM (neuronal networks) or are there other ...
1
vote
0answers
13 views
Gradient boosting Regression with zero-inflated outcome
I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
1
vote
0answers
40 views
Gradient boosting - Learning rate decay
Learning rate decay is a very used technique for training neural networks. Common beliefs are that:
an initially large learning rate accelerates training or helps the network escape local minima;
...
0
votes
1answer
27 views
What is the selection criteria to choose between XGBoost and Random Forest
I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when ...
1
vote
1answer
52 views
XGBoost multiclassification interpreting predicted probabilities
Let's consider an example. I have patient level data, their symptoms, reading from various medical tests. Based on that, I have built a binary classifier given patient data to classify if they are ...
0
votes
0answers
31 views
XGBoost Python API - performance by API, R vs Python inconsistencies, GPU memory and verbosity
I'm combining a few questions together as I feel that it could benefit others.
XGBoost API
Is there a performance boost (training time or accuracy) when you use the learning API vs the Scikit-Learn ...
6
votes
1answer
120 views
What is query id (“qid”) in XGBoost
In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet:
...
1
vote
2answers
39 views
Weights for unbalanced classification
I'm working with an unbalanced classification problem, in which the target variable contains:
np.bincount(y_train)
array([151953, 13273])
i.e. ...
1
vote
0answers
20 views
Classification Model showing different accuracy for SAME data?
This is my first post here, so kindly pardon any commonplace errors.
So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
1
vote
1answer
45 views
xgboost classifier predicted negative probabilities
I'm using XGBoost for a binary classification problem. There is no negative label, only 1 and 0.
I tunned the hyperparameters using Bayesian optimization then tried to train the final model with the ...
1
vote
1answer
14 views
scale_pos_weight using XGBoost's Learning API
I see it is possible to add a weight for unbalanced problems in XGBoost's Scikit-Learn API through scale_pos_weight. Does it have an equivalent in the Learning API? If not,
is there a reason behind ...
5
votes
2answers
167 views
XGBOOST - different result between train_test_split and manually splitting
I am trying to train XGBOOST model.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y)
when I'm using ...
1
vote
1answer
39 views
Interpretable xgboost - Calculate cover feature importance
When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance.
There are same parameters in the xgb api sucha as: weight, gain, cover, ...
3
votes
1answer
58 views
Small number of estimators in gradient boosting
I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
-1
votes
1answer
19 views
Machine Learning Out of test data forecast (XGBoost, ANN)
I see a lot of applications for machine learning techniques applied to time series. Unfortunately almost all kernels with XGBoost or ANN stop short in creating an actual forecast. The achieve a great ...
0
votes
0answers
27 views
Multiclass XGBoost train with num classes = 2
I have a tagged csv file with 5 calsses.
I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2.
The model I received seems to ...
2
votes
1answer
25 views
Each time the result of XGBoost is different
I'm running xgboost and I fix the seed number
but the result is different for each time when I rerun the xgboost with same dataset
...
1
vote
0answers
17 views
1
vote
0answers
17 views
Multi-label classification yielding too much unlabeled rows
I am performing multi-label classification with xgboost + OneVsRestClassifier from sklearn. ...
1
vote
1answer
25 views
1
vote
0answers
21 views
1
vote
2answers
16 views
is there cross validation for xgb classification for multi labels?
is there cross validation for xgb classification for multi labels?
I have been search but can not find any cross validation for xgb classifier
is using cross validation for xgb or xgb classifier ...
1
vote
1answer
17 views
is there any rule to apply pca to the imbalance data? [closed]
Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost)
I used multiclass imbalance data to pca
but the log-loss accuracy getting decrease
any theoritical background of this?
0
votes
0answers
10 views
Getting bad predictions for high true values of target variable
I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
1answer
37 views
XGBoost: How to set the probability threshold for multi class classification
I am using the XGBoost for classification of text data.
There are the 3 different classes in the training dataset.
...
0
votes
1answer
15 views
order of features importance after make_column_transformer and pipeline
I have a data preparation and model fitting pipeline that takes a dataframe (X_trn) and uses the ‘make_column_transformer’ and ‘Pipeline’ functions in sklearn to prepare the data and fit XGBRegressor.
...
1
vote
1answer
30 views
order of features for model tuning vs model fitting
Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
4
votes
1answer
152 views
ROC AUC score is much less than average cross validation score
Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
5
votes
1answer
104 views
Why gradient boosting uses sampling without replacement?
In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
1
vote
2answers
33 views
Can an entire data frame be used as a prediction variable?
I am attempting to use XGBoost in R to train a model that predicts a fixed number of target variables using all data from previous dates, as well as the two categorical variables (...
1
vote
1answer
27 views
How to check for “statistical significance” of categorical feature in black box models
Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
2
votes
1answer
50 views
How to calculate joint feature contribution for XGBoost Classifier in python?
I referred to http://savvastjortjoglou.com/intrepretable-machine-learning-nfl-combine.html#Joint-Feature-Contributions this beautiful document to research about joint feature contibutions. But this ...
4
votes
1answer
66 views
XGBoost: how to adjust the probabilities of a binary classifier to match training data?
Training and testing data have around 1% positives, but the model predicts only around 0.1% as positives.
The model is an xgboost classifier.
I’ve tried calibration but it didn’t improve much. I ...
0
votes
0answers
86 views
SHAP Kernel explainer for my pipeline model
I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer.
The data contains different types of ...
0
votes
1answer
150 views
DataFrame.dtypes error when training model using xgboost
I am getting a DataFrame.dtypes error while following the last steps of this tutorial:
https://www.hackerearth.com/practice/machine-learning/machine-learning-projects/python-project/tutorial/.
Here ...
1
vote
0answers
28 views
is it possible (and/or logical) to set feature importance for xgboost?
If I understand tree based methods correctly, it would be better for more important features to be toward the top of the tree.
Is there a way I can dictate this in xgboost? Similar to how I can ...
1
vote
3answers
38 views
Categorical variables with multiple entries transformed to entity embedding
I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction.
Some ...
2
votes
0answers
30 views
How to tune the hyperparameters of XGBoost and RF? [closed]
How to tune the hyperparameters of XGBoost and RF in python?
There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv?
...
4
votes
1answer
195 views
How are samples selected from training data in Xgboost
In Random Forest, each tree is not fed with the full batch of training data, only a sample.
How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
3
votes
1answer
180 views
How the algorithms such as Xgboost and LightGBM fills the NAN (missing data)?
How the algorithms such as Xgboost and LightGBM fills the NAN ?
I know that they learn how to fill it but how they do it is my question and what is they method of learning how to fill the NAN?
1
vote
1answer
146 views
XGBoost Feature Importance, Permutation Importance, and Model Evaluation Criteria
I have built an XGBoost classification model in Python on an imbalanced dataset (~1 million positive values and ~12 million negative values), where the features are binary user interaction with web ...
1
vote
2answers
42 views
3
votes
1answer
201 views
How do I predict survival curves using xgboost?
The xgboost package enables survival modeling using parameter arguments: objective = "survival:cox" and eval_metric = "cox-nloglik".
The predict method for the resulting model only outputs risk ...
0
votes
1answer
35 views
How to pass linear regression weights to Xgboost regressor?
I'm trying to build an xgboost regressor or a catboost regressor for a task. I have a working linear regression model. I also trained an xgboost regressor model for the task but it was worse than the ...
1
vote
2answers
53 views
What does many low important feature indicate?
I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset
When I used Xgboost Feature Importance, I ...
1
vote
0answers
17 views
Prediction issue with xgboost custom loss
I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried ...
1
vote
0answers
33 views
Is copying parameter considered as plagarism?
So my friends and i are writing a kaggle assignment and the base code is written by me.
One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
1
vote
1answer
96 views
What is the difference between a regular Linear Regression model and xgboost with objective set to “reg:linear”?
As I understand it, a regular linear regression model already minimizes for squared error, which means that it is the theoretical best prediction for this metric. Does xgboost's "reg:linear" ...
0
votes
0answers
41 views
Multi-output classifier using Tensorflow/Keras on tabular data
I have been tasked with combining a several classifier models we have into one model using deep learning (or something else).
The reason for this is that, in future, it would be difficult to maintain ...
