Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
0
votes
1answer
14 views
Ordinal classification with xgboost
I am working in the problem where the dependent variables are ordered classes, such as bad, good, ...
0
votes
0answers
27 views
XGBoost regression
I run XGBoost regression with tree as base learner. I have over 400 variables and more than 30000000 samples. I have generated most important features and was surprised to see that one feature is ...
1
vote
1answer
29 views
What are the limitations while using XGboost algorithm? [on hold]
Will XGBoost pose any problem while dealing with categorical variables with more than 2 levels.
For example, occupation variable can have values like doctor, engineer, lawyer, data scientist, farmer e....
0
votes
0answers
10 views
Using feature vectors from imagenet to train xgboost (vs a standard Conv net)?
I am planning to use feature vectors generated from imagenet to train an xgboost model. This is as opposed to training a standard convolutional network with the same image set.
This is because we ...
1
vote
1answer
34 views
inbalanced dataset with 3 classes xgboost scale_pos_weight parameter
xgboost classifier states the use of parameter scale_pos_weight for 2-class problem.
i have highly imbalanced dataset with 3 classes.
classes '1' and '-1' are very rare (~1% of dataset) and class '0'...
1
vote
1answer
25 views
XGBoost most important features appear in multiple trees multiple times
I am fitting xgboost model (scala-spark) to my dataset of transactions. I have about 2 millions of transactions in my training set which is highly unbalanced with a ratio of positive/negative<0.001 ...
1
vote
1answer
24 views
What does the limit of xgboost max_depth=1 represent?
In my mind, this means that each tree just takes one feature, and produces a step function based upon it.
In the limit of n_estimators being very large and max_depth=1, does xgboost become a linear ...
2
votes
0answers
38 views
Target transformation for tree models
Can anybody explain why/if target variable transformations could help when dealing with tree based models?
I've seen this excellent reply which explains quite well why it shouldn't affect if ...
2
votes
1answer
26 views
Should I create metafeatures for my XGBoost training set?
Say I've got two (not necessarily independent) features A and B for my dataset. Should I create metafeatures from them? say for ...
0
votes
0answers
77 views
Survival analysis in xgboost
Does anyone know the expected format for survival analysis data in xgboost?
The documentation states that you can select survival:cox as a learning objective but I ...
1
vote
1answer
80 views
LightGBM - Why Exclusive Feature Bundling (EFB)?
I'm currently studying GBDT and started reading LightGBM's research paper: https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf
In section 4. they explain ...
0
votes
1answer
42 views
Why xgboost can not deal with this simple sentence case?
There is only 1 feature dim. But the result is unreasonable. The code and data is below. The purpose of the code is to judge whether the two sentences are the same.
In fact, the final input to the ...
6
votes
1answer
31 views
What is meant by Distributed for a gradient boosting library?
I am checking out XGBoost documentation and it's stated that XGBoost is an optimized distributed gradient boosting library.
What is meant by distributed?
Have a nice day
1
vote
2answers
64 views
Both train and test error are decreasing in XGBoost iterations
I have an issue with training an XGBoost classifier in a sence that both train and test error only decrease throughout more iterations (num_boost_round) even if I use 1000 num boost rounds and 10 ...
0
votes
1answer
84 views
does xgb multi-class require one-hot encoding?
I was trying an xgboost from python with a multiclass single-label problem and assumed the label can be an integer indicating my class (as opposed to eg one-hot) .
...
0
votes
0answers
68 views
xgboost load_model killing kernel
I am trying to load a saved xgboost model, which consistently kills my kernel.
bst = xgb.Booster()
filename = 'is_latin_gbm_883_5a.pkl'
bst.load_model(filename)
...
1
vote
0answers
14 views
Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?
Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
2
votes
1answer
86 views
Can I use xgboost on a dataset with 1000 rows for classification problem?
I have used all types of classification algorithms on my dataset yet I couldn't improve my score no matter how I try.
So I've read about Xgboost classifier. So I was wondering is it practical to use ...
0
votes
0answers
35 views
Fitting a gradient boosting regression model on residuals of a linear model
I'm working on time series forecasting of electrical consumption. What you need to know is that there are tons of categorical features, so I opted for a regressive model (gradient boosting) instead of ...
0
votes
0answers
97 views
Calculating gradient and hessian for a custom loss function to use in xgboost
I want to use a cost function which rewards true positives, true negatives, and penalizes false positives and false negatives differently.
Something like the one below
...
0
votes
0answers
43 views
0
votes
1answer
114 views
Improving prediction accuracy with XGBoost
I have a 32x20 matrix for which I am trying to use XGBoost (Regression). I am looping through rows to produce an out of sample forecast. I'm surprised that XGBoost only returns an out of sample ...
1
vote
0answers
29 views
What would be the equivalent of R's mboost in Python?
I am looking for the Python equivalent of R's mboost package ( mboost ).
Would that be xgboost?
3
votes
1answer
116 views
Handling unbalanced datasets with XG boosting
Suppose you want to model (predict) a rare disease, and you use the parameter "pos scale weight" as a hyperparameter in XG boost . For example I have 20 times more positive cases, can I then use pos ...
2
votes
0answers
191 views
Credit scoring using scorecardpy with XGBoost
I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
1
vote
0answers
127 views
Comparing XGBR with CatBoost performance
I saw on a CatBoost site that it supposed to outperform any other boosted training model and decided to try it myself on a Kaggle's https://www.kaggle.com/c/house-prices-advanced-regression-techniques....
3
votes
1answer
228 views
Sensitivity analysis of a machine learning model
Let’s say I have a set of input variables (A, B, C and D)...
2
votes
1answer
412 views
Xgboost performs significantly worse than Random Forest
I have a dataset of 3500 observations x 70 features which is my training set and I also have a dataset of 600 observations x 70 features which is the test set.
The target is to classify observations ...
1
vote
1answer
74 views
boosting an xgboost classifier with another xgboost classifier using different sets of features
What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
1
vote
0answers
110 views
xgboost gain vs kolmogorov smirnov
After running xgboost model with:
objective = 'binary:logistic'
eval_metric = 'logloss'
I have a group of 3 variables that have the highest values of gain. Now, ...
0
votes
1answer
58 views
Impact of sparse features on tree-based models
Say you have a highly imbalanced binary classification problem.
Some of the features are binary features, where they're false most of the time, but when they're true they tend to be highly predictive (...
0
votes
0answers
19 views
Visualize strengths and weaknesses of a sample
Let's say I'm trying to predict an apartment price.
So, I have a lot of labeled data, where on each apartment I have features that could affect the price like:
city
street
floor
year built
...
0
votes
0answers
20 views
How to interpret a max_depth very small of my xgboost classifier
I have a binary classification problem.
I optimize max_depth and min_child_weight by GridSearchCV (nfold = 5, fold size ~500).
The parameter range varies for max_depth from 2 to 30 and for ...
0
votes
1answer
437 views
How to show progress of sklearn.multioutput.MultiOutputRegressor and XGBRegressor?
Is it possible to show the training progress of the MultiOutputRegressor in sklearn? When a huge dataset is processed, my program runs a long time and I have no clue how long it will take. I have ...
1
vote
0answers
187 views
how does XGBoost's exact greedy split finding algorithm determine candidate split values for different feature types?
Based on the paper by Chen & Guestrin (2016)
"XGBoost: A Scalable Tree Boosting System", XGBoost's "exact split finding algorithm enumerates over all the possible splits on all the features to ...
0
votes
0answers
132 views
XGboost - different AUC score when using R and Python
I had a project in R which I used XGboost in R and got AUC of 74%. I needed to transfer the project to python, I used the same dataframes: train and test as I used in R. I used XGboost (XGBClassifier ...
1
vote
2answers
159 views
Minimum number of samples to train XGBoost without overfitting
When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron.
Is there a similar rule of thumb for XGBoost, ...
0
votes
0answers
69 views
How to know feature values through the response values of trained XGBoost/CatBoost
I have boosting model which has very good approximation of some process (production of some product). I want to know which feature values leads to certain amount of a product, for example which values ...
2
votes
1answer
93 views
get_dump() leaf value and AUC
I have used Xgboost fitted a model with AUC around 0.73 and I printed out my last booster:
...
0
votes
0answers
136 views
Clarification about Normalized Discounted Cumulative Gain (NDCG) together with Regression for Ranking?
I want to rank products from 1 to M, my dataset looks like this:
...
0
votes
0answers
262 views
how to use label weights for pairwise ranking in xgboost?
As far as I know, to train learning to rank models, you need to have three things in the dataset:
label or relevance
group or query id
feature vector
For example, the Microsoft Learning to Rank ...
0
votes
0answers
21 views
Decision Tree in Regression - Sensitive amount of Splits in a Node
When creating a Decision Tree in Random Forest or in Xgboost a node makes different splits for each feature.
In case a feature can have just three possible values: 1, 2 and 3. The node will generate ...
1
vote
1answer
589 views
XGBoost: Quantifying Feature Importances
I need to quantify the importance of the features in my model. However, when I use XGBoost to do this, I get completely different results depending on whether I use the variable importance plot or the ...
1
vote
0answers
39 views
Regression Decision Tree - Normalize or Split into Ranges a continuos feature
I have in my dataset a feature named distances which ranges goes from 200 to 12000 (more or less). Since the other features have got values under 50 I need to do ...
2
votes
2answers
919 views
Python XGBoost killing kernel
My Jupyter notebook's python kernel keeps dying when attempting to train an XGBoost logistic classifier. Previously, I have run all of the following code successfully. Presently, there are issues. ...
0
votes
0answers
302 views
All probabilities in multiclass xgboost prediction the same
I train a model with objective multi:softprob on 80 datapoints and evaluate using just 20. I have a few thousand classes, and of the 20 evaluation datapoints, each assigns each of my classes the same ...
1
vote
2answers
69 views
How can machine learning algorithms solve this particular problem?
Let's think of a case of an e-commerce website which lists products for sale. Now a person can come on a particular product page and decide to add it to the shopping cart or not. If we look at it as a ...
1
vote
0answers
115 views
eta and learning_rate different in xgboost
I am creating a classification model using xgboost in python. I am using different eta values to check its effect on the model. My code is-
...
0
votes
0answers
112 views
Dropping one of the one-hot encoded columns for Gradient Boost Methods/Decision Trees?
If I have the categorical variable like favorite_color and it has unique values red, green, ...
0
votes
2answers
868 views
R langauge how to create xgb.DMatrix object from data frame (newbe)
In R, how does one create an xgb.DMatrix object from an R data frame?
