Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
383
questions
1
vote
1answer
15 views
1
vote
0answers
16 views
0
votes
1answer
9 views
is there cross validation for xgb classification for multi labels?
is there cross validation for xgb classification for multi labels?
I have been search but can not find any cross validation for xgb classifier
is using cross validation for xgb or xgb classifier ...
1
vote
1answer
16 views
is there any rule to apply pca to the imbalance data? [closed]
Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost)
I used multiclass imbalance data to pca
but the log-loss accuracy getting decrease
any theoritical background of this?
0
votes
0answers
10 views
Getting bad predictions for high true values of target variable
I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
1answer
22 views
XGBoost: How to set the probability threshold for multi class classification
I am using the XGBoost for classification of text data.
There are the 3 different classes in the training dataset.
...
0
votes
1answer
11 views
order of features importance after make_column_transformer and pipeline
I have a data preparation and model fitting pipeline that takes a dataframe (X_trn) and uses the ‘make_column_transformer’ and ‘Pipeline’ functions in sklearn to prepare the data and fit XGBRegressor.
...
1
vote
1answer
25 views
order of features for model tuning vs model fitting
Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
4
votes
1answer
134 views
ROC AUC score is much less than average cross validation score
Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
0
votes
1answer
17 views
Model stability [closed]
I trained a model and its performance varies as shown in the plot below. Can I safely say my model is stable on the basis of this information?
Thanks in advance.
4
votes
0answers
45 views
Why gradient boosting uses sampling without replacement?
In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
1
vote
2answers
31 views
Can an entire data frame be used as a prediction variable?
I am attempting to use XGBoost in R to train a model that predicts a fixed number of target variables using all data from previous dates, as well as the two categorical variables (...
1
vote
1answer
23 views
How to check for “statistical significance” of categorical feature in black box models
Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
1
vote
1answer
23 views
How to calculate joint feature contribution for XGBoost Classifier in python?
I referred to http://savvastjortjoglou.com/intrepretable-machine-learning-nfl-combine.html#Joint-Feature-Contributions this beautiful document to research about joint feature contibutions. But this ...
-1
votes
0answers
43 views
What are the pros and cons of XGBoost?
I just found some pros and cons for tree methods in general (https://medium.com/syncedreview/tree-boosting-with-xgboost-why-does-xgboost-win-every-machine-learning-competition-ca8034c0b283).
What are ...
0
votes
0answers
40 views
SHAP Kernel explainer for my pipeline model
I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer.
The data contains different types of ...
0
votes
1answer
56 views
DataFrame.dtypes error when training model using xgboost
I am getting a DataFrame.dtypes error while following the last steps of this tutorial:
https://www.hackerearth.com/practice/machine-learning/machine-learning-projects/python-project/tutorial/.
Here ...
1
vote
0answers
26 views
is it possible (and/or logical) to set feature importance for xgboost?
If I understand tree based methods correctly, it would be better for more important features to be toward the top of the tree.
Is there a way I can dictate this in xgboost? Similar to how I can ...
1
vote
3answers
27 views
Categorical variables with multiple entries transformed to entity embedding
I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction.
Some ...
2
votes
0answers
26 views
How to tune the hyperparameters of XGBoost and RF? [closed]
How to tune the hyperparameters of XGBoost and RF in python?
There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv?
...
4
votes
1answer
148 views
How are samples selected from training data in Xgboost
In Random Forest, each tree is not fed with the full batch of training data, only a sample.
How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
3
votes
1answer
74 views
How the algorithms such as Xgboost and LightGBM fills the NAN (missing data)?
How the algorithms such as Xgboost and LightGBM fills the NAN ?
I know that they learn how to fill it but how they do it is my question and what is they method of learning how to fill the NAN?
1
vote
1answer
68 views
XGBoost Feature Importance, Permutation Importance, and Model Evaluation Criteria
I have built an XGBoost classification model in Python on an imbalanced dataset (~1 million positive values and ~12 million negative values), where the features are binary user interaction with web ...
1
vote
2answers
38 views
3
votes
1answer
80 views
How do I predict survival curves using xgboost?
The xgboost package enables survival modeling using parameter arguments: objective = "survival:cox" and eval_metric = "cox-nloglik".
The predict method for the resulting model only outputs risk ...
0
votes
1answer
26 views
How to pass linear regression weights to Xgboost regressor?
I'm trying to build an xgboost regressor or a catboost regressor for a task. I have a working linear regression model. I also trained an xgboost regressor model for the task but it was worse than the ...
1
vote
2answers
52 views
What does many low important feature indicate?
I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset
When I used Xgboost Feature Importance, I ...
1
vote
0answers
16 views
Prediction issue with xgboost custom loss
I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried ...
1
vote
0answers
33 views
Is copying parameter considered as plagarism?
So my friends and i are writing a kaggle assignment and the base code is written by me.
One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
1
vote
1answer
58 views
What is the difference between a regular Linear Regression model and xgboost with objective set to “reg:linear”?
As I understand it, a regular linear regression model already minimizes for squared error, which means that it is the theoretical best prediction for this metric. Does xgboost's "reg:linear" ...
0
votes
0answers
29 views
Multi-output classifier using Tensorflow/Keras on tabular data
I have been tasked with combining a several classifier models we have into one model using deep learning (or something else).
The reason for this is that, in future, it would be difficult to maintain ...
0
votes
0answers
33 views
0
votes
0answers
48 views
Examples of the use of xgboost for recommender systems?
Are there any state-of-the-art implementations of xgboost in recommender systems? I'm looking for GitHub implementations but also papers that discuss this.
I've only found this paper https://...
2
votes
0answers
53 views
weighted quantile sketch in xgboost
I am unable to understand what is weighted quantile sketch in xgboost. Can anyone help me give an intuitive understanding of this?
1
vote
0answers
253 views
1
vote
1answer
41 views
Lime Explainer: ValueError: training data did not have the following fields
I'm attempting to gather ID level drivers from my XGBoost classification model using LIME and I'm running into some odd errors. I'm using this link as a reference.
Here is the overall code that I'm ...
0
votes
1answer
106 views
Unable to import xgboost 0.9
When running
import xgboost as xgb
print(xgb.__version__)
I am getting the following error:
...
5
votes
1answer
90 views
What is Pruning & Truncation in Decision Trees?
Pruning & Truncation
As per my understanding
Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to ...
4
votes
1answer
82 views
SHAP value can explain right?
I face a problem with using SHAP value to interpret the Tree-based model.
(https://github.com/slundberg/shapsd)
First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
57 views
Prediction error without having a true value
Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
1
vote
1answer
28 views
best way to regularize gradient boosting regressor?
i am testing gradient boosting regressor from sklearn for time series prediction on noisy data (currency markets).
https://scikit-learn.org/stable/modules/generated/sklearn.ensemble....
0
votes
1answer
17 views
sklearn.feature_selection vs xgboost feature_importances?
sklearn.feature_selection vs xgboost feature_importances
Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
0
votes
0answers
25 views
Why XGBoost regressor predicts behavior but not the amplitude?
I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling).
I found out that it works very well for predicting the ...
2
votes
1answer
45 views
Average of importance gain for a categorical variable
Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc).
I train an XGBoost model on a numeric ...
1
vote
1answer
31 views
XGBClassifier: make the output of predict_proba ascending regarding a specific feature
I'm trying to build a classifier using Xgboost on some high dimensional data, the problem I'm having is that I have the prior knowledge that the output probabilities should be ascending regarding a ...
0
votes
0answers
23 views
Should the LightGBM score match the regularization?
If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
2
votes
0answers
29 views
SHAP Explanations in case of repeated train/test split
I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that ...
1
vote
2answers
28 views
Adding extra variables to XGboost model is worsening the train and test accuracy
I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split.
However, when I am adding two new features , the ...
1
vote
0answers
26 views
Feature Vectors representation
I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
0
votes
0answers
190 views
XGBoost for multi-label image classification
I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
