Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

  7. current community

    your communities

    more stack exchange communities

    company blog
Data Science

Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

231 questions
info Newest Frequent Votes Active Unanswered
1
vote
0answers
33 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
0
votes
1answer
19 views

How to reach continue training in xgboost

I read the paper but found nothing talking about how to implement incremental learning. Can someone share some basic or deep knowledge? not in coding way. I know how to write code snippet to train ...
0
votes
0answers
8 views

Multiple XGBoost models or just 1 for a cetain type of category?

I am building a model to predict, say house prices. Within my data I have sales and rentals. The Y variable is the price of either the sales or rentals. I also have ...
0
votes
1answer
18 views

How to favour a particular class during classification using XGBoost?

I am using a simple XGBoost model to classify 2 classes (0 and 1) in a binary context. In case of the original data, the 0 is the majority class and 1 the minority class. The thing which is happening ...
0
votes
0answers
16 views

Making a model to predict the error of another model

So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to ...
0
votes
0answers
4 views

Install graphiz on AWS Sagemaker [migrated]

I'm on a Jupyter notebook using Python3 and trying to plot a tree with code like this: ...
1
vote
1answer
26 views

Balancing XGboost still skews towards the majority class

I have unbalanced dataset for multiclass classification and I tried to use the class weights option in XGboost and the classifier still tends to favor the majority class. I am not sure if I need to ...
0
votes
0answers
17 views

XGBoost becomes unstable when predicting more than ~300 classes

I'm using the Python implementation of XGBoosts (version 0.80) XGBoostClassifier to predict one of a large number of classes. My feature data consists of a sparse matrix of ~10M rows, ~5k columns, ...
1
vote
0answers
24 views

How does L1 Regularization work in lightGBM

From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance. My question is that, when the objective is L1 loss/...
0
votes
0answers
17 views

Using logLoss as metric function for highly unbalanced dataset

ihave an highly unbalanced dataset and the caret pacjage only allows me to select accuracy or kappa as performance metric. Is it correct to use a mlogloss function to compute model performance? Do you ...
0
votes
0answers
18 views

When to use SVR over other regression models

I am confused about when to use Support Vector Regression over other models like Random Forest Regression and XGBoost. I expected XGBoost to give the best prediction score(r2_score) for my regression ...
0
votes
0answers
16 views

XGBoost feature significance and feature importance

In a regression model it is possible to judge at a specified significance level (often alpha = 5%) whether a variable has a significant influence on the target attribute. With XGBoost, you can use ...
0
votes
0answers
46 views

why do we need other machine learning algorithms when we have xgboost

I am new to machine learning and learning it through an online course. For solving a regression problem I see various algorithms such as Linear Regression, Random Forest, SVR etc. Fianlly I was ...
0
votes
0answers
18 views

How xgboost runs in parallel?

Xgboost is one of the boosting techniques which iterates the accuracy and reduce errors in sequential way. But xgboost is named for running parallel and thus achieves fast computation. Can anyone ...
0
votes
2answers
26 views

How to ensure same encoding pattern?

I created a XGBRegressor model with certain encoded 'object' dtypes in the data. Now if I want to run the model with new set of data which is freshly encoded it's giving wrong predictions. How to ...
0
votes
0answers
10 views

Xgboost prediction on Sagemaker's side

Good afternoon. Simple .fit({'train': s3_train_data, 'validation': s3_validation_data}) is called on an Estimator to train it by pulling data from s3 as s3_input (hence the s3_train_data and ...
0
votes
0answers
16 views

Validition score while training lower than on final model with xgboost

I have 3 three classes, but my metric is auc, so I have customer eval metric: ...
1
vote
2answers
31 views

How to deal with overestimation of small values and underestimation of high values in XGBoost?

I'm running XGBoost to predict prices on a cars dataset, I was wondering what alternatives are there for this kind of problem ...
0
votes
0answers
37 views

Xgboost rank:ndcg learning per group or for all dataset

I'm trying to implement xgboost with an objective of rank:ndcg I want the target to be between 0-3. In my data for most of the groups, there is only 1 event per ...
1
vote
0answers
12 views

xgboost or lightgbm to handle Binomial problems

I have a dataset containing a column of trials, a column of successes and other features; and, obviously, I can generate a probability column. I would like to use gradient boosting methods (like ...
1
vote
2answers
21 views

Time of trainig vs time of prediction, which one is used during classification algorithms comparison?

I need to use many algorithms for making a binary classification, such as Logistic regression, SVM, XGBoost, CatBoost, ... I get an interesting improvement but All of those algorithms (except LR) take ...
0
votes
3answers
104 views

when can xgboost or catboost be better then Logistic regression?

I need to improve the prediction result of an algorithm that is already programmed based on logistic regression ( for binary classification). I tried to use XGBoost and CatBoost (with default ...
0
votes
1answer
31 views

python: why add new features of image to be worse result with xgboost

there are about 93 rows of data with features, two classes label. And, there are about 49 one-hot value features, and there are about 10 features continuous value. I split the data randomly by train ...
0
votes
0answers
34 views

Is plotting gain of XGBoost trees useful?

I am working on a XGBoost model for fraud detection (2 class classification) using XGBoost v0.7 on Spark. I am looking at different aspects of the model to find important features but also to ...
0
votes
1answer
63 views

Ordinal classification with xgboost

I am working in the problem where the dependent variables are ordered classes, such as bad, good, ...
0
votes
0answers
33 views

XGBoost regression

I run XGBoost regression with tree as base learner. I have over 400 variables and more than 30000000 samples. I have generated most important features and was surprised to see that one feature is ...
1
vote
1answer
53 views

What are the limitations while using XGboost algorithm? [closed]

Will XGBoost pose any problem while dealing with categorical variables with more than 2 levels. For example, occupation variable can have values like doctor, engineer, lawyer, data scientist, farmer e....
1
vote
0answers
15 views

Using feature vectors from imagenet to train xgboost (vs a standard Conv net)?

I am planning to use feature vectors generated from imagenet to train an xgboost model. This is as opposed to training a standard convolutional network with the same image set. This is because we ...
2
votes
1answer
145 views

inbalanced dataset with 3 classes xgboost scale_pos_weight parameter

xgboost classifier states the use of parameter scale_pos_weight for 2-class problem. i have highly imbalanced dataset with 3 classes. classes '1' and '-1' are very rare (~1% of dataset) and class '0'...
1
vote
1answer
53 views

XGBoost most important features appear in multiple trees multiple times

I am fitting xgboost model (scala-spark) to my dataset of transactions. I have about 2 millions of transactions in my training set which is highly unbalanced with a ratio of positive/negative<0.001 ...
1
vote
1answer
42 views

What does the limit of xgboost max_depth=1 represent?

In my mind, this means that each tree just takes one feature, and produces a step function based upon it. In the limit of n_estimators being very large and max_depth=1, does xgboost become a linear ...
2
votes
0answers
43 views

Target transformation for tree models

Can anybody explain why/if target variable transformations could help when dealing with tree based models? I've seen this excellent reply which explains quite well why it shouldn't affect if ...
2
votes
1answer
42 views

Should I create metafeatures for my XGBoost training set?

Say I've got two (not necessarily independent) features A and B for my dataset. Should I create metafeatures from them? say for ...
1
vote
1answer
144 views

LightGBM - Why Exclusive Feature Bundling (EFB)?

I'm currently studying GBDT and started reading LightGBM's research paper: https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf In section 4. they explain ...
0
votes
1answer
53 views

Why xgboost can not deal with this simple sentence case?

There is only 1 feature dim. But the result is unreasonable. The code and data is below. The purpose of the code is to judge whether the two sentences are the same. In fact, the final input to the ...
6
votes
1answer
35 views

What is meant by Distributed for a gradient boosting library?

I am checking out XGBoost documentation and it's stated that XGBoost is an optimized distributed gradient boosting library. What is meant by distributed? Have a nice day
1
vote
2answers
153 views

Both train and test error are decreasing in XGBoost iterations

I have an issue with training an XGBoost classifier in a sence that both train and test error only decrease throughout more iterations (num_boost_round) even if I use 1000 num boost rounds and 10 ...
0
votes
1answer
168 views

does xgb multi-class require one-hot encoding?

I was trying an xgboost from python with a multiclass single-label problem and assumed the label can be an integer indicating my class (as opposed to eg one-hot) . ...
2
votes
0answers
25 views

Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
2
votes
1answer
170 views

Can I use xgboost on a dataset with 1000 rows for classification problem?

I have used all types of classification algorithms on my dataset yet I couldn't improve my score no matter how I try. So I've read about Xgboost classifier. So I was wondering is it practical to use ...
0
votes
1answer
177 views

Improving prediction accuracy with XGBoost

I have a 32x20 matrix for which I am trying to use XGBoost (Regression). I am looping through rows to produce an out of sample forecast. I'm surprised that XGBoost only returns an out of sample ...
1
vote
0answers
36 views

What would be the equivalent of R's mboost in Python?

I am looking for the Python equivalent of R's mboost package ( mboost ). Would that be xgboost?
3
votes
1answer
248 views

Handling unbalanced datasets with XG boosting

Suppose you want to model (predict) a rare disease, and you use the parameter "pos scale weight" as a hyperparameter in XG boost . For example I have 20 times more positive cases, can I then use pos ...
3
votes
1answer
319 views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
1
vote
0answers
164 views

Comparing XGBR with CatBoost performance

I saw on a CatBoost site that it supposed to outperform any other boosted training model and decided to try it myself on a Kaggle's https://www.kaggle.com/c/house-prices-advanced-regression-techniques....
3
votes
1answer
418 views

Sensitivity analysis of a machine learning model

Let’s say I have a set of input variables (A, B, C and D)...
3
votes
2answers
701 views

Xgboost performs significantly worse than Random Forest

I have a dataset of 3500 observations x 70 features which is my training set and I also have a dataset of 600 observations x 70 features which is the test set. The target is to classify observations ...
1
vote
1answer
118 views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
1
vote
0answers
150 views

xgboost gain vs kolmogorov smirnov

After running xgboost model with: objective = 'binary:logistic' eval_metric = 'logloss' I have a group of 3 variables that have the highest values of gain. Now, ...

1 2 3 4 5