Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
565
questions
0
votes
0answers
9 views
XGBoost:Logistic poor performance with Scaled and PCA data
Working on a data set similar to fraud but not monetary transactions. Here are the steps that I have taken on the modeling side:
Convert Some of the categorical into numerical (One hot encode)
Over ...
0
votes
0answers
20 views
Is my approach for a loss function that adds more importance to negative samples reasonable?
For my current project I'm using XGBoost Regression to predict values y_pred with mean = 0 and std = 1. I want my model to place more emphasis on predicting samples right, where the true value y_true ...
1
vote
0answers
6 views
How to understand Xgboost model dump
Noticed that spark xgboost does not have a API trees_to_dataframe() as that in Python API, I am trying to parse the getModelDump ...
0
votes
0answers
24 views
XGBoost failing on highly imbalanced data!
I am working on a classification problem, where I am trying to predict a fraud login. The data is highly imbalanced i.e.
0 = non fraud logins , 1 = fraud logins
0 : 4538076
1 : 365
I have been trying ...
0
votes
1answer
19 views
Random Forest but keep only leaves with impurities below a threshold
Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine?
In other words, if I set min ...
0
votes
0answers
25 views
Train/ Test split on small dataset along with SMOTE
I have a binary classification imbalanced dataset with 1000 samples ( 15% of class 1, 85% of the rest). My main goal is to build a robust classifier using the following approach. Wanted to know if ...
1
vote
1answer
19 views
Tree complexity in xgboost
According to xgboost paper, regularization is given by:
$$\Omega(f) = \gamma T + \lambda || w||^2$$
where $\gamma$ is the complexity of a tree (i.e., number of leaves in the tree).
The parameter ...
0
votes
1answer
35 views
Is it possible to do Normalization before Xgboost?
Currently I am working on a project which uses Xgboost Regression.
Before putting data into model, I implemented Normalization, the accuracy significantly increased compared with without Normalization....
0
votes
0answers
13 views
Techniques for Ordinal Classification/Regression with Gradient Boosted Trees
I did some research on how to run ordinal classification decision trees (such as lightgbm, xgboost), and found these articles to be helpful. Both use a k-1 binary classification technique to output ...
2
votes
0answers
15 views
XGBoost incremental training for big datasets
I am trying to train an XGBoost model on a quite big dataset (tens of GB, almost a hundred).
I have been trying to use some libraries such as Dask to deal with this problem, without any success due to ...
0
votes
0answers
10 views
What's the difference between multiclass categorical crossentropy, mlogloss and multi:softprob?
As far as I understand, an objective is something I'm trying to optimize and an evaluation statistic is something I use to look for overfitting. I stumbled upon 4 losses that seem to be the same, but ...
0
votes
0answers
16 views
Is it possible that shap feature importance result will be more accurate than gain?
XGboost build the boosted tree in the following way:
Each level of each tree (the phase of selecting the next feature with conditional value) selected according ...
0
votes
2answers
30 views
Why is XGBClassifier in Python outputting different feature importance values with the same data across different repetitions?
I am fitting an XGBClassifier to a small dataset (32 subjects) and find that if I loop through the code 10 times the feature importances (gain) assigned to the features in the model varies slightly.
I ...
0
votes
1answer
20 views
How do you do 1-vs-rest classifiers in XGBoost Library (Not Sklearn)?
I am working with a very large dataset that would benefit from using training continuation with the xgb_model parameter in ...
2
votes
0answers
18 views
Highly correlated Features problematic when using column subsampling?
I've found some answers online regarding highly correlated Features in Boosting Trees, e.g. here or here.
The consensus seems to be that it's rather problematic interpretation-wise and not in regard ...
1
vote
1answer
30 views
one hot encoding target variable in tree and non tree (knn) methods
I am learning about label encoders, one hot encoding etc applied to datasets for classification via KNN and XGBoost type trees. However, I am a bit confused as to whether the target variable should be ...
0
votes
2answers
30 views
select hyperparameters using Latin hypercube sampling (LHS) from a large matrix/grid of parameter combinations
I have a matrix with each row corresponds to a hyperparameter for the XGBoost model. There are seven parameters to tune in XGBoost (as shown below: nrounds/iterations, max_depth, eta, gamma, ...
0
votes
1answer
26 views
Why is KNN better at K-Fold Cross Validation than XGBoost or Random Forest?
I've been running K-Fold cross validation multiple times for KNN, random forest and XGBoost.
KNN can complete sklearn's cross_val_score, so much faster consistently.
They all use the same preprocessed ...
0
votes
1answer
37 views
High Recall but too low Precision result in imbalanced data
I was training a model using XGBoost Classifier on heavy imbalanced data base with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature ...
0
votes
0answers
8 views
Model performance in different snapshots varying
I am trying to solve this problem. A medical representative needs to visit some doctors' clinics and for that a model will generate probability scores for visiting a clinic. I ma using a tree based ...
0
votes
0answers
7 views
How to handle features containing strings in XGBoost in AWS Sagemaker
How can i handle the string containing spaces and colons as a feature for my xgboost classifier model?
AWS Sagemaker requires the input in csv format, I don't know how to convert the string to the ...
0
votes
0answers
15 views
XGBoost: is increasing gamma same as feature selection by average gain?
Since gamma limits splits unless they meet a minimum gain threshold, isn't that the same thing as removing features that have low average gain? Both will results in splits with higher average gains.
I ...
0
votes
0answers
8 views
How to customize a objective funtion in xgboost?
I am new to xgboost and not so good at math.
I want to use a self defined objective function, here is the expression:
...
0
votes
1answer
7 views
XG Boost Regression to model Log of Dependent Variable
I'm working on a data set that has continuous dependent variable. I used XG Boost to model the dependent variable. However, when I transformed the dependent variable by applying Log transformation and ...
0
votes
1answer
25 views
What are some good models to complement XGBOOST in stacking?
What are some good models to complement XGBOOST via stacking in typical Kaggle datascience competition? I realize XGBoost with well-tuned hyperparamters are generally quite good already.
1
vote
2answers
21 views
How to deal with the difference in the range of Dependent variable in Train and Test data
I am training a XGBoost model for predicting number of applications and the minimum number of applications in training data is 40 and the maximum number of applications is 2000, while in test set ...
6
votes
1answer
98 views
What is the best way (cheapest / fastest option) to train an model on massive dataset (400GB+, 100m rows x 200 columns)?
I have a 400GB data set that I want to train a model on. What is the cheapest method to train this model? The options I can think of so far are:
AWS instance with massive RAM and train CPU (slow, but ...
0
votes
0answers
18 views
XGBoost hardware requirements. CPU vs. memory
I am working on predictive models with ML using very roughly 10-50 million records (currently testing with less) and around 10 explanatory variables per model.
When outlining hardware requirements for ...
0
votes
0answers
9 views
Getting low/very uncalibrated predictions on new data for regression using xgbtree method in train() function in R's caret package
I recently tried creating a regression model (imagine a model with the target variable being an integer with values 10-110) using xgboost (method = 'xgbTree') with caret. The model trains successfully,...
0
votes
0answers
7 views
Tree Based Classification (XGBoost, LightGBM, etc) - Features from embeddings for sparse features?
I'm wondering if there is a possibility from using embeddings as inputs for tree based classification models?
For example we have a field called type of food, and ...
0
votes
1answer
38 views
What is “Missing” in output of plot_tree API of XGBoost
What this "Missing" term means here at each node after split in Image? and also what is at leaf, is this means prediction value? I converted Output variable to 1 and 0.
I tried searching on ...
0
votes
1answer
46 views
Model Dump Parser (like XGBFI) for LightGBM and CatBoost
Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
0
votes
1answer
30 views
Hypertune xgboost to dealing with imbalanced dataset
My training data has extremely class imbalanced {0:872525,1:3335} with 100 features. I use xgboost to build classification model with bayessian optimisation to hypertune the model in range
...
0
votes
1answer
26 views
Consequences of using XGBoost regressor for small dataset(< 500 rows)
I am using XGBoost regressor to train my model for 322 rows of data and the train and test split is as follows: ((257, 9), (257,), (65, 9), (65,))
I am using the ...
1
vote
1answer
31 views
Re-training regression model on covid data [closed]
I am trying to re-train a regression model (XGB regressor) which was used in the pre-covid times (Feb 2020).
The dependent variable for the model is the number of bookings done, and due to covid, the ...
3
votes
1answer
23 views
Is addition of a random control variable a good idea, to ensure that results are correct?
I'm dealing with a complex dataset with many patients who have a condition, and various qualities about these patients. I'm trying to determine patient outcome based on patient qualities.
I'm using <...
1
vote
0answers
17 views
XGBoost with deep trees
I've been exploring the use of XGBoost in many different applications. Up to now, I always find the best results with shallow trees (from 1 to 3 levels), with the rest of the parameters very dependent ...
0
votes
0answers
20 views
Cost Function Binary Classification
I have imbalance dataset for binary classification problem. I want to create a custom cost function that takes into account not only the actual class and probability, but another variable "...
0
votes
1answer
24 views
Dealing with unseen data/categories in machine learning models for stream data
I want to build a machine learning model (xgb and lgbm) that has to handle streaming data on a weekly basis. The models are trained on a bi-weekly basis. The data includes order information and I want ...
0
votes
0answers
13 views
How are missing values treated in XGB RF Classifier?
I was exploring Random Forest Classifier in XGBoost listed here : https://xgboost.readthedocs.io/en/latest/python/python_api.html
I was wondering how the missing values will be handled in this ...
0
votes
1answer
25 views
100% Accuracy on test dataset using a previous developed model oputput
My dependent variable is a probability that is sourced from someone else's classification model. I am using this probability as a dependent variable as I don't have the actual data. On building an ...
0
votes
0answers
13 views
How to incrementally train xgb with additional features?
I found this thread similar to my question:
How to reach continue training in xgboost
However, I'm looking for a way to incrementally train my xgb model with ...
0
votes
0answers
5 views
How to explain ANN can predict much larger output values (e.g., y>2.5) when it was only trained with small output values (y>=2.5)
I have trained models with both ANN and XGBoost. I am wondering that whether ANN has the ability to predict much larger output values (e.g., $y>2.5)$ when it was only trained with small output ...
0
votes
0answers
13 views
Transforming binary data for decision trees
I have binary columns in my dataset (20) e.g. hot_weather, discount (y or no), where in each case 1 = yes no = 0. I am using this data on tree based methods.
It is a regression problem and my RMSE is ...
0
votes
0answers
24 views
Time Series Modelling or Simple regression or something else
PROJECT: I am working on an e-commerce site where digital products can run out so there is need to reorder them 72h before they run out (reordering them sooner is not a problem but having notification ...
0
votes
1answer
22 views
How does tree-based algorithms handle linearly combined features?
While I am aware that tree-based algorithms (e.g., DT, RF, XGBoost) are 'immune' to multi-collinearity, how do they handle linearly combined features?
For example, is there is any additional value or ...
0
votes
0answers
13 views
Timeseries param tuning using XGBOOST
I am using xgboost for timeseries forecasting of a certain attribute while including seasonal features.Trained on nearly 4 years of data and tested on the last month. My rmse is as below :
Hyper ...
0
votes
2answers
33 views
Using Transaction Amount to Guide Learning in an Fraud Detection Machine Learning Model
I am currently using transaction amount as a feature in an XGBoost classification model designed to identify fraudulent transactions. Furthermore, transaction amount is bounded for this problem ...
0
votes
0answers
28 views
kmeans cluster results for use as xgboost feature
I am curious if kmeans clusters can be used as xgboost features, along with the original features?
Specifically for features X and labels Y can we:
Split into X_train, y_train & X_test, y_test
...
1
vote
0answers
46 views
XGBoost: Typical gamma and min_child_weight range
What is the typical accepted range of gamma and min_child_weight parameters for the XGBoost algorithm?
Is the range of min_child_weight correlated with the number of feature or samples in the training ...
