Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
15 views

feature importance and xgboost?

Let say I got feature importance for xgclassifier ...
1
vote
0answers
16 views

is there metric 'multi_logloss' for xgb crassifier?

lgb has the log_loss metric ...
0
votes
1answer
9 views

is there cross validation for xgb classification for multi labels?

is there cross validation for xgb classification for multi labels? I have been search but can not find any cross validation for xgb classifier is using cross validation for xgb or xgb classifier ...
1
vote
1answer
16 views

is there any rule to apply pca to the imbalance data? [closed]

Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost) I used multiclass imbalance data to pca but the log-loss accuracy getting decrease any theoritical background of this?
0
votes
0answers
10 views

Getting bad predictions for high true values of target variable

I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
1answer
22 views

XGBoost: How to set the probability threshold for multi class classification

I am using the XGBoost for classification of text data. There are the 3 different classes in the training dataset. ...
0
votes
1answer
11 views

order of features importance after make_column_transformer and pipeline

I have a data preparation and model fitting pipeline that takes a dataframe (X_trn) and uses the ‘make_column_transformer’ and ‘Pipeline’ functions in sklearn to prepare the data and fit XGBRegressor. ...
1
vote
1answer
25 views

order of features for model tuning vs model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
4
votes
1answer
134 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
0
votes
1answer
17 views

Model stability [closed]

I trained a model and its performance varies as shown in the plot below. Can I safely say my model is stable on the basis of this information? Thanks in advance.
4
votes
0answers
45 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
1
vote
2answers
31 views

Can an entire data frame be used as a prediction variable?

I am attempting to use XGBoost in R to train a model that predicts a fixed number of target variables using all data from previous dates, as well as the two categorical variables (...
1
vote
1answer
23 views

How to check for “statistical significance” of categorical feature in black box models

Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
1
vote
1answer
23 views

How to calculate joint feature contribution for XGBoost Classifier in python?

I referred to http://savvastjortjoglou.com/intrepretable-machine-learning-nfl-combine.html#Joint-Feature-Contributions this beautiful document to research about joint feature contibutions. But this ...
-1
votes
0answers
43 views

What are the pros and cons of XGBoost?

I just found some pros and cons for tree methods in general (https://medium.com/syncedreview/tree-boosting-with-xgboost-why-does-xgboost-win-every-machine-learning-competition-ca8034c0b283). What are ...
0
votes
0answers
40 views

SHAP Kernel explainer for my pipeline model

I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer. The data contains different types of ...
0
votes
1answer
56 views

DataFrame.dtypes error when training model using xgboost

I am getting a DataFrame.dtypes error while following the last steps of this tutorial: https://www.hackerearth.com/practice/machine-learning/machine-learning-projects/python-project/tutorial/. Here ...
1
vote
0answers
26 views

is it possible (and/or logical) to set feature importance for xgboost?

If I understand tree based methods correctly, it would be better for more important features to be toward the top of the tree. Is there a way I can dictate this in xgboost? Similar to how I can ...
1
vote
3answers
27 views

Categorical variables with multiple entries transformed to entity embedding

I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction. Some ...
2
votes
0answers
26 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python? There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv? ...
4
votes
1answer
148 views

How are samples selected from training data in Xgboost

In Random Forest, each tree is not fed with the full batch of training data, only a sample. How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
3
votes
1answer
74 views

How the algorithms such as Xgboost and LightGBM fills the NAN (missing data)?

How the algorithms such as Xgboost and LightGBM fills the NAN ? I know that they learn how to fill it but how they do it is my question and what is they method of learning how to fill the NAN?
1
vote
1answer
68 views

XGBoost Feature Importance, Permutation Importance, and Model Evaluation Criteria

I have built an XGBoost classification model in Python on an imbalanced dataset (~1 million positive values and ~12 million negative values), where the features are binary user interaction with web ...
3
votes
1answer
80 views

How do I predict survival curves using xgboost?

The xgboost package enables survival modeling using parameter arguments: objective = "survival:cox" and eval_metric = "cox-nloglik". The predict method for the resulting model only outputs risk ...
0
votes
1answer
26 views

How to pass linear regression weights to Xgboost regressor?

I'm trying to build an xgboost regressor or a catboost regressor for a task. I have a working linear regression model. I also trained an xgboost regressor model for the task but it was worse than the ...
1
vote
2answers
52 views

What does many low important feature indicate?

I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset When I used Xgboost Feature Importance, I ...
1
vote
0answers
16 views

Prediction issue with xgboost custom loss

I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried ...
1
vote
0answers
33 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
1
vote
1answer
58 views

What is the difference between a regular Linear Regression model and xgboost with objective set to “reg:linear”?

As I understand it, a regular linear regression model already minimizes for squared error, which means that it is the theoretical best prediction for this metric. Does xgboost's "reg:linear" ...
0
votes
0answers
29 views

Multi-output classifier using Tensorflow/Keras on tabular data

I have been tasked with combining a several classifier models we have into one model using deep learning (or something else). The reason for this is that, in future, it would be difficult to maintain ...
0
votes
0answers
33 views

Changing objective functions with penalty term in XGboost in R

I basically have something like: ...
0
votes
0answers
48 views

Examples of the use of xgboost for recommender systems?

Are there any state-of-the-art implementations of xgboost in recommender systems? I'm looking for GitHub implementations but also papers that discuss this. I've only found this paper https://...
2
votes
0answers
53 views

weighted quantile sketch in xgboost

I am unable to understand what is weighted quantile sketch in xgboost. Can anyone help me give an intuitive understanding of this?
1
vote
0answers
253 views
1
vote
1answer
41 views

Lime Explainer: ValueError: training data did not have the following fields

I'm attempting to gather ID level drivers from my XGBoost classification model using LIME and I'm running into some odd errors. I'm using this link as a reference. Here is the overall code that I'm ...
0
votes
1answer
106 views

Unable to import xgboost 0.9

When running import xgboost as xgb print(xgb.__version__) I am getting the following error: ...
5
votes
1answer
90 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to ...
4
votes
1answer
82 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
57 views

Prediction error without having a true value

Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
1
vote
1answer
28 views

best way to regularize gradient boosting regressor?

i am testing gradient boosting regressor from sklearn for time series prediction on noisy data (currency markets). https://scikit-learn.org/stable/modules/generated/sklearn.ensemble....
0
votes
1answer
17 views

sklearn.feature_selection vs xgboost feature_importances?

sklearn.feature_selection vs xgboost feature_importances Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
0
votes
0answers
25 views

Why XGBoost regressor predicts behavior but not the amplitude?

I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling). I found out that it works very well for predicting the ...
2
votes
1answer
45 views

Average of importance gain for a categorical variable

Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc). I train an XGBoost model on a numeric ...
1
vote
1answer
31 views

XGBClassifier: make the output of predict_proba ascending regarding a specific feature

I'm trying to build a classifier using Xgboost on some high dimensional data, the problem I'm having is that I have the prior knowledge that the output probabilities should be ascending regarding a ...
0
votes
0answers
23 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
2
votes
0answers
29 views

SHAP Explanations in case of repeated train/test split

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that ...
1
vote
2answers
28 views

Adding extra variables to XGboost model is worsening the train and test accuracy

I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split. However, when I am adding two new features , the ...
1
vote
0answers
26 views

Feature Vectors representation

I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
0
votes
0answers
190 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...

1 2 3 4 5 8