Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

SHAP Kernel explainer for my pipeline model

I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer. The data contains different types of ...
0
votes
1answer
27 views

how to solve an error when training model using xgboost

I am following the tutorial at https://www.hackerearth.com/practice/machine-learning/machine-learning-projects/python-project/tutorial/. I am almost at the end of it but I started getting an error. ...
1
vote
0answers
26 views

is it possible (and/or logical) to set feature importance for xgboost?

If I understand tree based methods correctly, it would be better for more important features to be toward the top of the tree. Is there a way I can dictate this in xgboost? Similar to how I can ...
1
vote
3answers
25 views

Categorical variables with multiple entries transformed to entity embedding

I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction. Some ...
2
votes
0answers
25 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python? There are several methods to tune hyperparameteres of XGBoost and RF such as Bayesian Optimization and meta learning and gridseachcv? ...
3
votes
1answer
107 views

How are samples selected from training data in Xgboost

In Random Forest, each tree is not fed with the full batch of training data, only a sample. How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
3
votes
1answer
48 views

How the algorithms such as Xgboost and LightGBM fills the NAN (missing data)?

How the algorithms such as Xgboost and LightGBM fills the NAN ? I know that they learn how to fill it but how they do it is my question and what is they method of learning how to fill the NAN?
1
vote
1answer
40 views

XGBoost Feature Importance, Permutation Importance, and Model Evaluation Criteria

I have built an XGBoost classification model in Python on an imbalanced dataset (~1 million positive values and ~12 million negative values), where the features are binary user interaction with web ...
3
votes
1answer
43 views

How do I predict survival curves using xgboost?

The xgboost package enables survival modeling using parameter arguments: objective = "survival:cox" and eval_metric = "cox-nloglik". The predict method for the resulting model only outputs risk ...
0
votes
1answer
21 views

How to pass linear regression weights to Xgboost regressor?

I'm trying to build an xgboost regressor or a catboost regressor for a task. I have a working linear regression model. I also trained an xgboost regressor model for the task but it was worse than the ...
1
vote
2answers
49 views

What does many low important feature indicate?

I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset When I used Xgboost Feature Importance, I ...
1
vote
0answers
14 views

Prediction issue with xgboost custom loss

I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried ...
1
vote
0answers
33 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
1
vote
1answer
43 views

What is the difference between a regular Linear Regression model and xgboost with objective set to “reg:linear”?

As I understand it, a regular linear regression model already minimizes for squared error, which means that it is the theoretical best prediction for this metric. Does xgboost's "reg:linear" ...
0
votes
0answers
21 views

Multi-output classifier using Tensorflow/Keras on tabular data

I have been tasked with combining a several classifier models we have into one model using deep learning (or something else). The reason for this is that, in future, it would be difficult to maintain ...
0
votes
0answers
23 views

Changing objective functions with penalty term in XGboost in R

I basically have something like: ...
0
votes
0answers
45 views

Examples of the use of xgboost for recommender systems?

Are there any state-of-the-art implementations of xgboost in recommender systems? I'm looking for GitHub implementations but also papers that discuss this. I've only found this paper https://...
2
votes
0answers
38 views

weighted quantile sketch in xgboost

I am unable to understand what is weighted quantile sketch in xgboost. Can anyone help me give an intuitive understanding of this?
1
vote
0answers
185 views
1
vote
1answer
33 views

Lime Explainer: ValueError: training data did not have the following fields

I'm attempting to gather ID level drivers from my XGBoost classification model using LIME and I'm running into some odd errors. I'm using this link as a reference. Here is the overall code that I'm ...
0
votes
1answer
74 views

Unable to import xgboost 0.9

When running import xgboost as xgb print(xgb.__version__) I am getting the following error: ...
5
votes
1answer
79 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to ...
4
votes
1answer
70 views

SHAP value can explain right?

I face a problem with using SHAP value to interpret the Tree-based model. (https://github.com/slundberg/shapsd) First, I have input around 30 features and I have 2 features that have high positive ...
0
votes
0answers
56 views

Prediction error without having a true value

Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
1
vote
1answer
22 views

best way to regularize gradient boosting regressor?

i am testing gradient boosting regressor from sklearn for time series prediction on noisy data (currency markets). https://scikit-learn.org/stable/modules/generated/sklearn.ensemble....
0
votes
1answer
16 views

sklearn.feature_selection vs xgboost feature_importances?

sklearn.feature_selection vs xgboost feature_importances Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
0
votes
0answers
24 views

Why XGBoost regressor predicts behavior but not the amplitude?

I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling). I found out that it works very well for predicting the ...
2
votes
1answer
42 views

Average of importance gain for a categorical variable

Suppose I have a set of M categorical variables, some of them with a different number of categories (for instance, var1 has five categories, var2 has three, etc). I train an XGBoost model on a numeric ...
1
vote
1answer
29 views

XGBClassifier: make the output of predict_proba ascending regarding a specific feature

I'm trying to build a classifier using Xgboost on some high dimensional data, the problem I'm having is that I have the prior knowledge that the output probabilities should be ascending regarding a ...
0
votes
0answers
23 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
2
votes
0answers
26 views

SHAP Explanations in case of repeated train/test split

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that ...
1
vote
2answers
28 views

Adding extra variables to XGboost model is worsening the train and test accuracy

I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split. However, when I am adding two new features , the ...
1
vote
0answers
25 views

Feature Vectors representation

I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
0
votes
0answers
142 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
1
vote
0answers
51 views

Hyperparameter Tuning using Bayesian Techniques

I've been looking into Bayesian optimization for hyperparameter tuning and trying to compare the results I get to those I get using different methods (random grid search). I came across this site, ...
4
votes
2answers
62 views

Can I specify the root node splitting feature in XGBoost?

Just what the title says. Suppose I know the feature that I want to be used for splitting for the root node in a tree model in XGBoost; is there a way for me to tell XGBoost to split on this feature ...
0
votes
0answers
20 views

Feature engineering ideas with dates, coordinates and other variables

I'm working on an ETA problem where I'm trying to estimate a time of arrival for a delivery. I have coordinates of pickup/destination, time of pick , infos about the rider, some other variables that ...
0
votes
0answers
73 views

Suggestion for handling specific missing data

I have data, that describes distance from given location to nearest object (e.g. school, shop etc). Because of performance reasons I couldn't scrape the data about objects, that are futher away than 2....
0
votes
0answers
12 views

Improve the results of imbalanced multi-classification multi-lables data

I have 10k rows of multi-classification (x1..x27,y), size of dataframe is: 28*10k and its ...
4
votes
2answers
116 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...
1
vote
1answer
121 views

How does XGBoost use softmax as an objective function?

I'm quite used to seeing functions like log-loss, RMSE, cross entropy as objective functions and it's easy to imagine why minimizing these would give us the best model. What's difficult to imagine is ...
0
votes
2answers
168 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
0
votes
2answers
348 views

What is the difference between gradient descent and gradient boosting? Are they interdependent on each other by any way?

What is the difference between gradient descent and gradient boosting? Are they interdependent on each other in any way ?
0
votes
1answer
66 views

XGBoost probability distribution tending towards the extreme

I am using an XGBoost classifier to make risk predictions, and I see that even if it has very good binary classification results, the probability outputs are mainly under $0.05$ or over $0.95$ (like ...
0
votes
1answer
29 views

Similarity of XGBoost models?

Is xgboost with n_estimators = 100 and learning_rate = 0.1, same as xgboost with n_estimators = 50 and learning_rate = 0.2 ?
1
vote
0answers
30 views

XGBoost speed issues

I'm trying to optimize the hyperparameters for XGBoost, thus needing to run it multiple times with different parameters. However the time needed to run single XGBoost with the parameters provided ...
0
votes
0answers
18 views

Data Transformation Tips for xgboost's XGBClassifier

I have this X_train and test distribution for the 4 features 'X', 'Y', 'TX' and 'TY'. I realize the range of the distribution is widely varying .. Can you suggest a good way to clean/ transform that ...
3
votes
2answers
206 views

XGBoost validation for number of trees

I have a simple Question: I am using XGBoost to classify some data: 1.) With 100 estimators I have following scores(roc_score): train_data : 98.5 validation_data : 97.2 2.) With 500 ...
0
votes
1answer
20 views

How to tell a boosting model that 2 features are related and should not be interpreted stand-alone?

I am using XGBoost for a machine learning model that learns from tabular data. XGBoost uses boosting method on decision trees. When I look at the decision-making logic of decision trees, I notice the ...