Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
17 views

Can you combine two xgboost models into one?

If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
1
vote
0answers
24 views

State of the Art/Research 2020 of Time Series Forecasting/Prediction

Im looking for the state of the art/research of time series data for forcasting/prediction. As far as im aware it is Extrem Gradient Boosting (XGBoost) or LSTM (neuronal networks) or are there other ...
1
vote
0answers
13 views

Gradient boosting Regression with zero-inflated outcome

I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
1
vote
0answers
40 views

Gradient boosting - Learning rate decay

Learning rate decay is a very used technique for training neural networks. Common beliefs are that: an initially large learning rate accelerates training or helps the network escape local minima; ...
0
votes
1answer
27 views

What is the selection criteria to choose between XGBoost and Random Forest

I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when ...
1
vote
1answer
52 views

XGBoost multiclassification interpreting predicted probabilities

Let's consider an example. I have patient level data, their symptoms, reading from various medical tests. Based on that, I have built a binary classifier given patient data to classify if they are ...
0
votes
0answers
31 views

XGBoost Python API - performance by API, R vs Python inconsistencies, GPU memory and verbosity

I'm combining a few questions together as I feel that it could benefit others. XGBoost API Is there a performance boost (training time or accuracy) when you use the learning API vs the Scikit-Learn ...
6
votes
1answer
120 views

What is query id (“qid”) in XGBoost

In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet: ...
1
vote
2answers
39 views

Weights for unbalanced classification

I'm working with an unbalanced classification problem, in which the target variable contains: np.bincount(y_train) array([151953, 13273]) i.e. ...
1
vote
0answers
20 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
1
vote
1answer
45 views

xgboost classifier predicted negative probabilities

I'm using XGBoost for a binary classification problem. There is no negative label, only 1 and 0. I tunned the hyperparameters using Bayesian optimization then tried to train the final model with the ...
1
vote
1answer
14 views

scale_pos_weight using XGBoost's Learning API

I see it is possible to add a weight for unbalanced problems in XGBoost's Scikit-Learn API through scale_pos_weight. Does it have an equivalent in the Learning API? If not, is there a reason behind ...
5
votes
2answers
167 views

XGBOOST - different result between train_test_split and manually splitting

I am trying to train XGBOOST model. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y) when I'm using ...
1
vote
1answer
39 views

Interpretable xgboost - Calculate cover feature importance

When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance. There are same parameters in the xgb api sucha as: weight, gain, cover, ...
3
votes
1answer
58 views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
-1
votes
1answer
19 views

Machine Learning Out of test data forecast (XGBoost, ANN)

I see a lot of applications for machine learning techniques applied to time series. Unfortunately almost all kernels with XGBoost or ANN stop short in creating an actual forecast. The achieve a great ...
0
votes
0answers
27 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
2
votes
1answer
25 views

Each time the result of XGBoost is different

I'm running xgboost and I fix the seed number but the result is different for each time when I rerun the xgboost with same dataset ...
1
vote
0answers
17 views

xgboost result is not stable [closed]

Here is my xgboost hyperparameters ...
1
vote
0answers
17 views

Multi-label classification yielding too much unlabeled rows

I am performing multi-label classification with xgboost + OneVsRestClassifier from sklearn. ...
1
vote
1answer
25 views

feature importance and xgboost?

Let say I got feature importance for xgclassifier ...
1
vote
0answers
21 views

is there metric 'multi_logloss' for xgb crassifier?

lgb has the log_loss metric ...
1
vote
2answers
16 views

is there cross validation for xgb classification for multi labels?

is there cross validation for xgb classification for multi labels? I have been search but can not find any cross validation for xgb classifier is using cross validation for xgb or xgb classifier ...
1
vote
1answer
17 views

is there any rule to apply pca to the imbalance data? [closed]

Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost) I used multiclass imbalance data to pca but the log-loss accuracy getting decrease any theoritical background of this?
0
votes
0answers
10 views

Getting bad predictions for high true values of target variable

I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
1answer
37 views

XGBoost: How to set the probability threshold for multi class classification

I am using the XGBoost for classification of text data. There are the 3 different classes in the training dataset. ...
0
votes
1answer
15 views

order of features importance after make_column_transformer and pipeline

I have a data preparation and model fitting pipeline that takes a dataframe (X_trn) and uses the ‘make_column_transformer’ and ‘Pipeline’ functions in sklearn to prepare the data and fit XGBRegressor. ...
1
vote
1answer
30 views

order of features for model tuning vs model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
4
votes
1answer
152 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
5
votes
1answer
104 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
1
vote
2answers
33 views

Can an entire data frame be used as a prediction variable?

I am attempting to use XGBoost in R to train a model that predicts a fixed number of target variables using all data from previous dates, as well as the two categorical variables (...
1
vote
1answer
27 views

How to check for “statistical significance” of categorical feature in black box models

Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
2
votes
1answer
50 views

How to calculate joint feature contribution for XGBoost Classifier in python?

I referred to http://savvastjortjoglou.com/intrepretable-machine-learning-nfl-combine.html#Joint-Feature-Contributions this beautiful document to research about joint feature contibutions. But this ...
4
votes
1answer
66 views

XGBoost: how to adjust the probabilities of a binary classifier to match training data?

Training and testing data have around 1% positives, but the model predicts only around 0.1% as positives. The model is an xgboost classifier. I’ve tried calibration but it didn’t improve much. I ...
0
votes
0answers
86 views

SHAP Kernel explainer for my pipeline model

I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer. The data contains different types of ...
0
votes
1answer
150 views

DataFrame.dtypes error when training model using xgboost

I am getting a DataFrame.dtypes error while following the last steps of this tutorial: https://www.hackerearth.com/practice/machine-learning/machine-learning-projects/python-project/tutorial/. Here ...
1
vote
0answers
28 views

is it possible (and/or logical) to set feature importance for xgboost?

If I understand tree based methods correctly, it would be better for more important features to be toward the top of the tree. Is there a way I can dictate this in xgboost? Similar to how I can ...
1
vote
3answers
38 views

Categorical variables with multiple entries transformed to entity embedding

I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction. Some ...
2
votes
0answers
30 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python? There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv? ...
4
votes
1answer
195 views

How are samples selected from training data in Xgboost

In Random Forest, each tree is not fed with the full batch of training data, only a sample. How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
3
votes
1answer
180 views

How the algorithms such as Xgboost and LightGBM fills the NAN (missing data)?

How the algorithms such as Xgboost and LightGBM fills the NAN ? I know that they learn how to fill it but how they do it is my question and what is they method of learning how to fill the NAN?
1
vote
1answer
146 views

XGBoost Feature Importance, Permutation Importance, and Model Evaluation Criteria

I have built an XGBoost classification model in Python on an imbalanced dataset (~1 million positive values and ~12 million negative values), where the features are binary user interaction with web ...
3
votes
1answer
201 views

How do I predict survival curves using xgboost?

The xgboost package enables survival modeling using parameter arguments: objective = "survival:cox" and eval_metric = "cox-nloglik". The predict method for the resulting model only outputs risk ...
0
votes
1answer
35 views

How to pass linear regression weights to Xgboost regressor?

I'm trying to build an xgboost regressor or a catboost regressor for a task. I have a working linear regression model. I also trained an xgboost regressor model for the task but it was worse than the ...
1
vote
2answers
53 views

What does many low important feature indicate?

I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset When I used Xgboost Feature Importance, I ...
1
vote
0answers
17 views

Prediction issue with xgboost custom loss

I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried ...
1
vote
0answers
33 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
1
vote
1answer
96 views

What is the difference between a regular Linear Regression model and xgboost with objective set to “reg:linear”?

As I understand it, a regular linear regression model already minimizes for squared error, which means that it is the theoretical best prediction for this metric. Does xgboost's "reg:linear" ...
0
votes
0answers
41 views

Multi-output classifier using Tensorflow/Keras on tabular data

I have been tasked with combining a several classifier models we have into one model using deep learning (or something else). The reason for this is that, in future, it would be difficult to maintain ...

1
2 3 4 5
9