Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
535
questions
0
votes
0answers
12 views
Consequences of using XGBoost regressor for small dataset(< 500 rows)
I am using XGBoost regressor to train my model for 322 rows of data and the train and test split is as follows: ((257, 9), (257,), (65, 9), (65,))
I am using the ...
1
vote
1answer
19 views
Re-training regression model on covid data
I am trying to re-train a regression model (XGB regressor) which was used in the pre-covid times (Feb 2020).
The dependent variable for the model is the number of bookings done, and due to covid, the ...
-1
votes
0answers
23 views
F1 score /Sorensen Dice / Bray Curtis Distance/ XGBoost / Custom objective function [closed]
My question is: How can I use the Bray Curtis distance as a custom objective function for XG Boost in R? Bray Curtis is a proxy for F1 score. For two vectors, Labels and Preds, the BC distance is Sum ...
3
votes
1answer
21 views
Is addition of a random control variable a good idea, to ensure that results are correct?
I'm dealing with a complex dataset with many patients who have a condition, and various qualities about these patients. I'm trying to determine patient outcome based on patient qualities.
I'm using <...
1
vote
0answers
15 views
XGBoost with deep trees
I've been exploring the use of XGBoost in many different applications. Up to now, I always find the best results with shallow trees (from 1 to 3 levels), with the rest of the parameters very dependent ...
0
votes
0answers
16 views
Cost Function Binary Classification
I have imbalance dataset for binary classification problem. I want to create a custom cost function that takes into account not only the actual class and probability, but another variable "...
0
votes
1answer
18 views
Dealing with unseen data/categories in machine learning models for stream data
I want to build a machine learning model (xgb and lgbm) that has to handle streaming data on a weekly basis. The models are trained on a bi-weekly basis. The data includes order information and I want ...
0
votes
0answers
10 views
How are missing values treated in XGB RF Classifier?
I was exploring Random Forest Classifier in XGBoost listed here : https://xgboost.readthedocs.io/en/latest/python/python_api.html
I was wondering how the missing values will be handled in this ...
0
votes
1answer
23 views
100% Accuracy on test dataset using a previous developed model oputput
My dependent variable is a probability that is sourced from someone else's classification model. I am using this probability as a dependent variable as I don't have the actual data. On building an ...
0
votes
0answers
10 views
How to incrementally train xgb with additional features?
I found this thread similar to my question:
How to reach continue training in xgboost
However, I'm looking for a way to incrementally train my xgb model with ...
0
votes
0answers
5 views
How to explain ANN can predict much larger output values (e.g., y>2.5) when it was only trained with small output values (y>=2.5)
I have trained models with both ANN and XGBoost. I am wondering that whether ANN has the ability to predict much larger output values (e.g., $y>2.5)$ when it was only trained with small output ...
0
votes
0answers
13 views
Transforming binary data for decision trees
I have binary columns in my dataset (20) e.g. hot_weather, discount (y or no), where in each case 1 = yes no = 0. I am using this data on tree based methods.
It is a regression problem and my RMSE is ...
0
votes
1answer
26 views
What data science models are best for explainability for regression problems?
Imagine you have to create a model to explain to stakeholders e.g. to predict price, weight, sales etc.. Which machine learning models offer the best in terms of explainability and interprability? ...
0
votes
0answers
23 views
Time Series Modelling or Simple regression or something else
PROJECT: I am working on an e-commerce site where digital products can run out so there is need to reorder them 72h before they run out (reordering them sooner is not a problem but having notification ...
0
votes
1answer
17 views
How does tree-based algorithms handle linearly combined features?
While I am aware that tree-based algorithms (e.g., DT, RF, XGBoost) are 'immune' to multi-collinearity, how do they handle linearly combined features?
For example, is there is any additional value or ...
0
votes
0answers
8 views
Timeseries param tuning using XGBOOST
I am using xgboost for timeseries forecasting of a certain attribute while including seasonal features.Trained on nearly 4 years of data and tested on the last month. My rmse is as below :
Hyper ...
0
votes
2answers
29 views
Using Transaction Amount to Guide Learning in an Fraud Detection Machine Learning Model
I am currently using transaction amount as a feature in an XGBoost classification model designed to identify fraudulent transactions. Furthermore, transaction amount is bounded for this problem ...
0
votes
0answers
18 views
kmeans cluster results for use as xgboost feature
I am curious if kmeans clusters can be used as xgboost features, along with the original features?
Specifically for features X and labels Y can we:
Split into X_train, y_train & X_test, y_test
...
1
vote
0answers
26 views
XGBoost: Typical gamma and min_child_weight range
What is the typical accepted range of gamma and min_child_weight parameters for the XGBoost algorithm?
Is the range of min_child_weight correlated with the number of feature or samples in the training ...
1
vote
1answer
30 views
Strong overfitting accompanying strong class imbalance
I'm training an xgboost binary classification model. The data I have is around 600k and positive is only 0.1% of it. I tried to use all overfitting prevention techniques xgboost has to offer (tune eta,...
0
votes
0answers
27 views
Confusion Matrix after XGBoost is showing positive as negative class
Please can you help me with confusion matrix. I've implemented the XGBoostClassifier. After fitting the model when I looked to the confusion matrix to view the performance on Test data. The confusion ...
1
vote
2answers
68 views
0
votes
1answer
35 views
Feature importance difference in two similar machine learning models
Situation 1:
I have trained a text classification model (Model 1) which gives me a probability of true class as X. I have also trained a classification model (Model 2) using only the categorical and ...
0
votes
0answers
38 views
XGBoost Regressor model reproducibility
I am running an XGBoost Regressor to predict electricity consumption (load) and further classify predicted values as peaks or not. As for dataset I started with hourly energy load data + hourly ...
2
votes
1answer
15 views
Output of evaluation metric for XGBoost - is it cumulative?
On the 10th boosting round for XGBoost, I get an MAP of 0.32 on the test data. Does that reflect the performance of just that 10th tree? Or the performance of all 10 trees combined that have been ...
0
votes
0answers
17 views
Is there a python package that includes decision tree structures that can be used with a genetic algorithm?
I'd like to use decision tree / forest but I need to use a special objective function that can't be differentiated and hence I can't use XGBOost etc. That leaves a genetic algorithm where I use the ...
1
vote
0answers
17 views
Is there a good alternative to XGBoost for learn to rank?
My problem with XGBoost is that when I load the train dataset into the XGBoost DMatrix, there is a memory spike that is unavoidable, and I can't get my dataset loaded into RAM without crashing first.
...
0
votes
1answer
13 views
XGBoost feature importance has all features but decision tree doesn't
I have used XGBoost to train a model with 400 features. My understanding is that since the max_depth is default at only 6, and 2^6 < 400, not all features will end up in the tree.
How come when I ...
1
vote
1answer
22 views
Does Gradient Boosting perform n-ary splits where n > 2?
I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
0
votes
0answers
18 views
sklearn GradientBoostingClassifier provides unstable predictions
I am using sklearn.ensemble.GradientBoostingClassifier to build a rather simple model which predicts probabilities.
I have simplified the problem to having only one numerical predictor and the outcome ...
1
vote
0answers
9 views
Prediction problem across a wide space
I have assembled a database of Clash Royale games in an attempt to understand the outcomes of various match-ups. The game is composed of an 8 card deck drawn from 102 cards. As you can see from the ...
0
votes
0answers
23 views
how to represent feature importance in xgboost in percentage?
I am looking for a way to represent the feature importance numbers in percentage. I read through articles and API documentation for XGboost in python and it gives me the feature importance score, ...
0
votes
0answers
9 views
How should I add reference key column to output after modeling is Complete?
0
I have created a csv of data with the following columns: (1) app_key (2) churn, (3)tenure
https://i.stack.imgur.com/NAlFF.png
I have performed the following code in order to drop app_key and churn
<...
1
vote
0answers
16 views
How can I use a validation set to tune the hyperparameters of an XGBClassifier?
I'm currently building a ranking model using an XGBClassifier.
I have training, testing, and validation sets. I want to use the validation set to tune the hyperparameters of the XGBClassifier before ...
0
votes
1answer
43 views
Why might my validation loss flatten out while my training loss continues to decrease?
In my effort to learn a bit more about data science I scraped some labeled data from the web and am trying to classify examples into one of three classes. I am running into a problem that, regardless ...
2
votes
1answer
118 views
From logistic regression to XGBoost - selecting features to run the model with
I have been asked to look at XGBoost (as implemented in R, and with a maximum of around 50 features) as an alternative to an already existing but not developed by me logistic regression model created ...
1
vote
2answers
38 views
Probabilities predicted by XGBoost
I looking at football data and trying to predict whether a goal will occur using xgboost with objective binary: logistic.
My data is 1:10 unbalanced with no goals being more dominant. I have used ...
2
votes
1answer
97 views
how to prevent machine crash while searching for hyper parameters of XGBoost with GridSearchCV
I am searching for best hyper parameters of XGBRegressor using GridSearchCV. Here is the code:
...
1
vote
1answer
71 views
Negative R2_score Bad predictions for my Sales prediction problem using LightGBM
My project involves trying to predict the sales quantity for a specific item across a whole year. I've used the LightGBM package for making the predictions. The params I've set for it are as follows:
<...
2
votes
1answer
117 views
How to approach All vs All classification problem
Let's say you are building a Star trek style medical tricorder which can diagnose any medical condition. It needs to be able to detect comorbidities where a patient has multiple conditions (e.g. ...
1
vote
1answer
29 views
how do tree based methods deal with missing feature columns?
all,
i have trained a model using xgboost. Some of the features are one hot encoded e.g. currency where it is either gbp or usd. it seems that when i output the feature importance gbp and usd were in ...
0
votes
0answers
30 views
Feature Selection before modeling with Boosting Trees
I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
0
votes
2answers
55 views
Hyperparameter tuning XGBoost
I'm trying to tune hyperparameters with bayesian optimization. It is a regression problem with the objective function: objective = 'reg:squaredlogerror'
$\frac{1}{2}[log(pred+1)-log(true+1)]^2$
My ...
0
votes
1answer
29 views
Bias variance tradeoff boosting (xgboost) vs random forest (randomized bagging) which to use when?
I was looking up differences between boosting an bagging and I see this quoted everywhere
If the classifier is unstable (high variance), then we should apply
Bagging. If the classifier is stable and ...
2
votes
1answer
37 views
Why Continous Variable Buckets Overfitting model
I have a continuous (high cardinal discrete) variable 'numInteractionPoints' in my dataset during training model - I binned this feature in order to avoid overffing , first top bar chart is from ...
0
votes
0answers
76 views
CNN with XGBoost as an output layer, is it better?
Context:
I have been experiencing with some Kaggle datasets to learn more about image classification. So, in this binary image classification task, I tried something that I thought would increase the ...
1
vote
1answer
19 views
How to quantify ‘compute cost’ of training of xgboost model?
I want to quantify compute cost of hyper-parameter search for xgboost model. One way can be to measure training time with one particular hyper parameter configuration chosen for training and use it as ...
1
vote
1answer
28 views
Tree-based algorithms and ordinal features
For tree-based methods (e.g., DT, Random Forest, Gradient boosting, etc.), does the conversion interval of an ordinal feature to continuous matter matters? (I can see why it matters for linear model, ...
1
vote
1answer
55 views
Python sklearn model.predict() gives me different results depending on the amount of data [closed]
I train my XGBoostClassifier().
If my testing set has:
0: 100
1: 884
It attempts to predict 210 1's. Around 147 are wrong (False positives) and 63 1's correctly ...
0
votes
0answers
22 views
How much does each tree of gradient boosting contribute to the global feature importance?
Let's say we are training a GBDT in the Titanic dataset. We have 3 trees in the GBDT.
You extract the first tree and calculate the feature importance (no matter if cover, gain...), and Age importance =...
