Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
583
questions
0
votes
0answers
10 views
What is the best way to create input data samples using in XGBoost for predicting number of next days that customer will come back to store
I'm building the tree-based model like a XGBoost to solve the problem about customer purchase cycle.
And I think, I will build 2 models which one is predicting the customer will come back to store in ...
2
votes
2answers
26 views
confused on "real score" vs "decision value" in classification trees
I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is:
A ...
1
vote
0answers
11 views
Classification for Ordinal labels - what tree-based methds can i use?
I have a label that has a natural ordering e.g. 0,1,2,3 where 0 is the worst activity measure and 3 is the best. For each label given by the model i need to also give the probability that it belongs ...
0
votes
0answers
15 views
lack of consistency in Bayesian optimization of xgboost's hyperparameters
I am trying to optimize the hyperparameters in an xgboost model using Bayesian optimization and the mlrmbo R package. The simplified code below seem to produce reasonable results, but the problem I ...
0
votes
0answers
14 views
User Churn Rate analysis - Binary classification
I have a dataset which has the logs of user clicks. This is a trail version(2 months) of the software. Users can use a special feature during this trail period to improve their sales. The number of ...
0
votes
0answers
9 views
Train and test data fixed during boosting?
I have question about boosting algorithm.
I know that boosting is a sequential process and it gives high weight to misclassification of previous model.
Then, its' train and test data are fixed through ...
1
vote
0answers
12 views
Why is the average prediction moving away from average response for a reg:gamma model
I'm predicting a response that I would typically model under a gamma distribution, with relatively simple paramters, I'm just using the default other than these:
learning_rate = 0.01
max_depth = 6
...
2
votes
1answer
41 views
Labeling and aggregating features issue
I am trying build a simple binary classifier (some tree based algorithm for now) and my training data will have features aggregated at the user level. So I'll have a unique records of each user. These ...
2
votes
2answers
134 views
Flask output not showing
I am trying to deploy a XGBClassifier model using flask. After giving the values to the relevant fields on the webpage, the ...
0
votes
0answers
18 views
Feature importance not providing score for each feature
I have a data frame of this shape (808616, 10744) and I use this model
...
0
votes
0answers
22 views
Should I use or tune `reg_lambda` or `reg_alpha` hyperparameters when using a tree booster in XGBoost
XGBoost has 3 types of boosters:
tree boosters (gbtree, dart)
linear booster (gbliner)
...
1
vote
1answer
26 views
How to interpret .get_booster().get_score(importance_type='weight') for XGBRegressor()
I am trying to do feature selection using XGRegressor(). I am doing this because I have many features to choose from over 4,000. Once I have a set of features I have a neural network I created to use ...
2
votes
1answer
40 views
Possible to use predict_proba without normalizing to 1?
I'm using xgboost multi-class classifier to predict a collection of things likely to fail. I want to run that prediction, and report anything that the classifier ...
0
votes
2answers
26 views
aggregation of feature importance
I have more of a conceptual question I was hoping to get some feedback on. I am trying to run a boosted regression ML model to identify a subset of important predictors for some clinical condition. ...
0
votes
1answer
90 views
Is feature importance in XGBoost or in any other tree based method reliable?
This question is quite long, if you know how feature importance to tree based methods works i suggest you to skip to text below the image.
Feature importance (FI) in tree based methods is given by ...
-1
votes
1answer
47 views
nested cross validation vs. train-test split
I am trying to understand the main benefits of conducting a nested cross-validation compared to a simpler train-test split. Let us say I would like to build a prediction model. I initially split my ...
0
votes
2answers
19 views
Concerns regarding small dataset with too many features
I have dataframe with 322 observations with 224 features. The observations has two classes, 0 or 1,which i'm trying to predict. class 0 has 168 observations and class 1 has 154 observations.
I was ...
0
votes
1answer
16 views
Gamma parameter in xgboost
As per the original paper on xgboost, the best split at a node is found by maximising the quantity below
$
\cal{L}_{\rm split} = \frac{1}{2} \sum \left [ \frac{G_L}{H_L + \lambda} + \frac{G_R}{H_R + \...
0
votes
0answers
9 views
Why is my validation score so much higher using TargetEncoder?
So I'm experimenting a bit with an XGBoost model & encoding the categorical variables using the target encoder from the category_encoders library. The code below shows how I split the dataset and ...
0
votes
0answers
17 views
How Random Forest and XGB 'Regressor' calculate feature importance
I am searching about how Random Forest and XGB 'regressor' calculate feature importance.
However, the most of discussion focus on Classifier.
I try to find out the answer in the official document but ...
0
votes
0answers
10 views
Is XGBoost active in the Gradient Boosting widget on the Orange platform?
When I open the Gradient Boosting widget xgboost is grayed out even though it is described in detail under the documentation section on Orange. The two options are Gradient Boosting (scikit-learn) and ...
0
votes
0answers
11 views
xgboost performance
XGBoostRegressor is not performing better than AdaBoostRegressor for the same set of parameters for some reason. Since my ...
2
votes
1answer
17 views
My own model trained on the full data is better than the best_estimator I get from GridSearchCV with refit=True?
I am using an XGBoost model to classify some data. I have cv splits (train, val) and a separate test set that I never use until the end.
I have used GridSearchCV to determine the best parameters and ...
0
votes
2answers
31 views
XgBoost given targets its only feature but fails when test targets are outside the range of training targets?
I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...
2
votes
0answers
26 views
Static ML model or Time-Series? How to model/predict a binary target when I have time variant features but most features are constant?
I have been working with Real World data from patients. I have a dataset with information about 10million patients; Collected over a span of varying duration (5 to 20 years).
What I am predicting is ...
2
votes
1answer
21 views
What if root of a such tree is pruned in xgboost?
Extreme Gradient Boosting stops to grow a tree if $\gamma$ is greater than impurity reduction given as eq (7) (see below) , what does happen if tree's root has a negative impurity? I think there is no ...
1
vote
1answer
160 views
XGBoost - Imputing Vs keeping NaN
What is the benefit of imputing numerical or categorical features when using DT methods such as XGBoost that can handle missing values? This question is mainly for when the values are missing not at ...
1
vote
1answer
12 views
XGboost with group-specific and individual-specific data
My dataset consists of a combination of two databases. One database consists of individual-level data on the characteristics and compensation of the top five executive officers of big American ...
1
vote
0answers
11 views
how could RFECV ever give the same score to all number of features selected?
I built an XGB and ran RFECV over 250 features. After an hour or so, I plotted the grid_scores_.
All numbers of features are within 0.02, as clearly visible on the y-axis.
To me this plot would ...
1
vote
0answers
22 views
Xgboost regressor : prediction overestimated or underestimated
Hi,
Recently I am working on a project about synthesizing wind turbine power by the other wind turbines, like the graph above , axis X is wind turbine A power and axis Y is wind turbine B power. All ...
1
vote
0answers
66 views
XGBoost regressor hyperparameter tuning with hyperopt leads to overfit
Using hyperopt to hyperparameter tuning on XGBoost regressor, I am receiving overfiting on the train set.
There is any suggestion how to solve it ?
I have used cross validation with ...
0
votes
0answers
13 views
XGBoost:Logistic poor performance with Scaled and PCA data
Working on a data set similar to fraud but not monetary transactions. Here are the steps that I have taken on the modeling side:
Convert Some of the categorical into numerical (One hot encode)
Over ...
0
votes
0answers
22 views
Is my approach for a loss function that adds more importance to negative samples reasonable?
For my current project I'm using XGBoost Regression to predict values y_pred with mean = 0 and std = 1. I want my model to place more emphasis on predicting samples right, where the true value y_true ...
1
vote
0answers
20 views
How to understand Xgboost model dump
Noticed that spark xgboost does not have a API trees_to_dataframe() as that in Python API, I am trying to parse the getModelDump ...
0
votes
0answers
32 views
XGBoost failing on highly imbalanced data!
I am working on a classification problem, where I am trying to predict a fraud login. The data is highly imbalanced i.e.
0 = non fraud logins , 1 = fraud logins
0 : 4538076
1 : 365
I have been trying ...
0
votes
1answer
20 views
Random Forest but keep only leaves with impurities below a threshold
Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine?
In other words, if I set min ...
0
votes
0answers
44 views
Train/ Test split on small dataset along with SMOTE
I have a binary classification imbalanced dataset with 1000 samples ( 15% of class 1, 85% of the rest). My main goal is to build a robust classifier using the following approach. Wanted to know if ...
1
vote
1answer
36 views
Tree complexity and gamma parameter in xgboost
According to xgboost paper, regularization is given by:
$$\Omega(f) = \gamma T + \lambda || w||^2$$
where $\gamma$ is the complexity of a tree (i.e., number of leaves in the tree).
The parameter ...
0
votes
1answer
45 views
Is it possible to do Normalization before Xgboost?
Currently I am working on a project which uses Xgboost Regression.
Before putting data into model, I implemented Normalization, the accuracy significantly increased compared with without Normalization....
0
votes
0answers
81 views
Techniques for Ordinal Classification/Regression with Gradient Boosted Trees
I did some research on how to run ordinal classification decision trees (such as lightgbm, xgboost), and found these articles to be helpful. Both use a k-1 binary classification technique to output ...
2
votes
0answers
79 views
XGBoost incremental training for big datasets
I am trying to train an XGBoost model on a quite big dataset (tens of GB, almost a hundred).
I have been trying to use some libraries such as Dask to deal with this problem, without any success due to ...
0
votes
0answers
95 views
What's the difference between multiclass categorical crossentropy, mlogloss and multi:softprob?
As far as I understand, an objective is something I'm trying to optimize and an evaluation statistic is something I use to look for overfitting. I stumbled upon 4 losses that seem to be the same, but ...
0
votes
0answers
26 views
Is it possible that shap feature importance result will be more accurate than gain?
XGboost build the boosted tree in the following way:
Each level of each tree (the phase of selecting the next feature with conditional value) selected according ...
1
vote
2answers
59 views
Why is XGBClassifier in Python outputting different feature importance values with the same data across different repetitions?
I am fitting an XGBClassifier to a small dataset (32 subjects) and find that if I loop through the code 10 times the feature importances (gain) assigned to the features in the model varies slightly.
I ...
0
votes
1answer
136 views
How do you do 1-vs-rest classifiers in XGBoost Library (Not Sklearn)?
I am working with a very large dataset that would benefit from using training continuation with the xgb_model parameter in ...
1
vote
1answer
62 views
one hot encoding target variable in tree and non tree (knn) methods
I am learning about label encoders, one hot encoding etc applied to datasets for classification via KNN and XGBoost type trees. However, I am a bit confused as to whether the target variable should be ...
0
votes
2answers
38 views
select hyperparameters using Latin hypercube sampling (LHS) from a large matrix/grid of parameter combinations
I have a matrix with each row corresponds to a hyperparameter for the XGBoost model. There are seven parameters to tune in XGBoost (as shown below: nrounds/iterations, max_depth, eta, gamma, ...
0
votes
1answer
54 views
Why is KNN better at K-Fold Cross Validation than XGBoost or Random Forest?
I've been running K-Fold cross validation multiple times for KNN, random forest and XGBoost.
KNN can complete sklearn's cross_val_score, so much faster consistently.
They all use the same preprocessed ...
3
votes
2answers
215 views
High Recall but too low Precision result in imbalanced data
I was training a model using XGBoost Classifier on a heavy imbalanced database with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature ...
