Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
55 views

GBM: small change in the trainset causes radical change in predictions

I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
0
votes
3answers
69 views

Can this problem be solved using deep learning?

I want to predict price of used cars. I have data like this: Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.? I used one hot approach for nominal features and ...
0
votes
1answer
1k views

What is out come cox regression in xgboost

I am running xgboost where objective is survival:cox and eval_metric is cox-nloglik. Y range from -800 to 800. However, predicted values are way to large in range from 10^3 to 10^13. I am not sure why ...
1
vote
1answer
32 views

xgboostclassifier prediction error after saving the model and restoring it

I have trained a xgboost model and during training, the prediction works fine. But if I stop the script and start a restoring script to restore and predict, then for the same test dataset I get every ...
1
vote
0answers
56 views

Variable Importance changes with oversampling

I am currently using Xgboost for a binary classification problem with highly imbalanced data in R. I have used oversampling to train the model. This worked well, now however it comes to measuring ...
1
vote
0answers
461 views

Bayesian optimization for a Light GBM Model

I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
1
vote
0answers
1k views

Custom loss function for XGBoost

I am looking for code to implement a custom loss function instead of just classification error, or cross entropy for gradient boosted classification trees. We are trying to model regime detection in ...
0
votes
1answer
83 views

XGBOOST CV() producing error

I am getting the following error while using xgboost.cv() (scikit-learn interface). I am working on a regression problem. Below is the code and trace. No idea why ...
2
votes
1answer
52 views

Training a model where each response in the observation data has a different known varience

I have a dataset where each response variable is the number of successes of N Bernoulli trials with N and p (the probability of success) being different for each observation. The goal is to train a ...
1
vote
2answers
449 views

Why does it not need to set test group when using 'rank:pairwise' in xgboost?

I'm new for learning-to-rank. I'm trying to learn the Learning to rank example provided by xgboost. I found that the core code is as follows in rank.py. ...
0
votes
0answers
33 views

Suggestion for model performance improvement for ML competition

I am working on highly imbalanced dataset and trying to increase accuracy(metric: roc_auc) of my model which is hovering around 82-83%. This is part of an internal ...
2
votes
2answers
722 views

Random forest vs. XGBoost vs. MLP Regressor for estimating claims costs

Context I'm building a (toy) machine learning model estimate the cost of an insurance claim (injury related). Aim is to teach myself machine learning by doing. I have settled on three algorithms to ...
3
votes
2answers
393 views

Hyper parameters tuning XGBClassifier

I am working on a highly imbalanced dataset for a competition. The training data shape is : (166573, 14) ...
10
votes
1answer
3k views

Lightgbm vs xgboost vs catboost

I've seen that in Kaggle competitions people are using lightgbms where they used to use xgboost. My question is: when would you rather use xgboost instead of lightgbm? What about catboost?
2
votes
0answers
241 views

Adjust class weights due to class imbalance and class importance Multi class classification XGBoost

With respect to this question and the answer given by @Esmailian, Would anyone be able to let me know if Class B has a higher importance or the positive class ( i.e. it needs to have a higher ...
1
vote
0answers
25 views

if new feature downgrade the score for xgboost what do I have to look at?

let say I'm predicting the housing price of Boston(kaggle). if I got some score x then I added new feature y_K if this new feature drop the score. what is wrong with this feature and what do I ...
0
votes
1answer
36 views

xgboost and linear regression new feature analysis

For linear regression, seems like a new feature has to be a linear relation with the target variable. But If you make the new feature for the Xgboost, what do you have to consider to make a new ...
3
votes
1answer
584 views

XGBRegressor hyperparameter optimization using xgb cv function

I am trying to optimize hyper parameters of XGBRegressor using xgb's cv function and bayesian optimization (using hyperopt package). Here is the piece of code I am using for the cv part. ...
1
vote
1answer
160 views

correct setting of eval_set in multiclass classification xgboost python , error is “ Check failed: preds.size() == info.labels_.size()”

i have a multiclass classification problem with 3 classes [-1,0,1] . i'd like to use eval_set in xgboost. but it fails with error: ...
0
votes
0answers
49 views

Training xgboost model with more data having different characteristic

I have trained my model for ECG data which has 8528 ECG files having length 30s and sample rate 300 so total file length in csv ...
1
vote
1answer
136 views
0
votes
0answers
68 views

XGBoost multiclass class balancing using weight parameter [duplicate]

I have three classes in the target variable with representation ratios of, class A:0.5 class B:0.3 ...
0
votes
2answers
30 views

Given a single discrete data set, how should I divide it into training data and test data?

I have a dataset in libSVM format consisting of 6000 entries, each with 5 indices, and each index has a binary value 1 or 2. Each of the 6000 entries has a label of ...
0
votes
1answer
765 views

XGBOOST (sklearn interface) REGRESSION error

I am trying to run a GRIDSEARCHCV (sklearn) on XGBRegressor. Documentation on the parameter says that if regression, then objective = reg:squarederror.(see https://...
1
vote
0answers
61 views

Why am I getting accuracy of Xgboost model 0.00%?

I am trying to build Job Recommender System using Deep Learning. dataset used From this dataset i have taken only users.tsv, user_history.tsv, apps.tsv and jobs.tsv to build a hybrid recommender ...
0
votes
2answers
189 views

Discrete Ordinal Classification with Probabilities

If I have classes 1, 2, 3 and 4. But, I also need the probability for each of the other classes. I'm currently using XGBoost for one-vs-rest classification, but that means we're losing information ...
0
votes
0answers
70 views

Reverse engineering on Xgboost model

I am doing experiments on https://www.physionet.org/challenge/2017/sources/ submission. I like one of the submission code, which use Xgboost to train the ...
0
votes
1answer
969 views
1
vote
0answers
57 views

How do I train Xgboost classifier for ECG Signal data?

I am testing https://www.physionet.org/challenge/2017/sources/ submission. I like one of the submission code, which use Xgboost to train the classifier. Training ...
4
votes
1answer
394 views

What happens if GBM parameters (e.g., learning rate) vary as the training progresses?

In neural networks there is an idea of a "learning rate schedule" which changes the learning rate as training progresses. This made me ask the question, what would be the impact of varying ...
3
votes
1answer
31 views

How do I create a data set that has a set of features for multiple options, with one option being the expected outcome?

Most datasets I see are: feature 1, feature 2, feature 3, outcome Where outcome is binary e.g. if they are cancer positive outcome will be 1 and 0 if they don't have cancer. How do I create a ...
1
vote
0answers
39 views

Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20) 0 : 91% 1 : 9% To build model on this dataset having class imbalance, I have compared ...
1
vote
0answers
48 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
0
votes
1answer
1k views

How to reach continue training in xgboost

I read the paper but found nothing talking about how to implement incremental learning. Can someone share some basic or deep knowledge? not in coding way. I know how to write code snippet to train ...
0
votes
0answers
23 views

Multiple XGBoost models or just 1 for a cetain type of category?

I am building a model to predict, say house prices. Within my data I have sales and rentals. The Y variable is the price of either the sales or rentals. I also have ...
0
votes
1answer
125 views

How to favour a particular class during classification using XGBoost?

I am using a simple XGBoost model to classify 2 classes (0 and 1) in a binary context. In case of the original data, the 0 is the majority class and 1 the minority class. The thing which is happening ...
0
votes
0answers
62 views

Making a model to predict the error of another model

So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to ...
2
votes
1answer
356 views

Balancing XGboost still skews towards the majority class

I have unbalanced dataset for multiclass classification and I tried to use the class weights option in XGboost and the classifier still tends to favor the majority class. I am not sure if I need to ...
0
votes
0answers
95 views

XGBoost becomes unstable when predicting more than ~300 classes

I'm using the Python implementation of XGBoosts (version 0.80) XGBoostClassifier to predict one of a large number of classes. My feature data consists of a sparse matrix of ~10M rows, ~5k columns, ...
1
vote
0answers
129 views

How does L1 Loss work in lightGBM

From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance. My question is that, when the objective is L1 loss/...
0
votes
0answers
118 views

Using logLoss as metric function for highly unbalanced dataset

ihave an highly unbalanced dataset and the caret pacjage only allows me to select accuracy or kappa as performance metric. Is it correct to use a mlogloss function to compute model performance? Do you ...
1
vote
0answers
22 views
0
votes
0answers
113 views

XGBoost feature significance and feature importance

In a regression model it is possible to judge at a specified significance level (often alpha = 5%) whether a variable has a significant influence on the target attribute. With XGBoost, you can use ...
0
votes
2answers
37 views

How to ensure same encoding pattern?

I created a XGBRegressor model with certain encoded 'object' dtypes in the data. Now if I want to run the model with new set of data which is freshly encoded it's giving wrong predictions. How to ...
1
vote
1answer
439 views

Using a combination of gradient boosting with LSTM for classification?

I am presently using an LSTM model to classify high dimensional tabular data which is not text/images (dimensions 21392x1970). I also tried XGBoost (Gradient boosting) in Python separately for the ...
0
votes
0answers
77 views

Validition score while training lower than on final model with xgboost

I have 3 three classes, but my metric is auc, so I have customer eval metric: ...
1
vote
2answers
184 views

How to deal with overestimation of small values and underestimation of high values in XGBoost?

I'm running XGBoost to predict prices on a cars dataset, I was wondering what alternatives are there for this kind of problem ...
1
vote
0answers
289 views

Xgboost rank:ndcg learning per group or for all dataset

I'm trying to implement xgboost with an objective of rank:ndcg I want the target to be between 0-3. In my data for most of the groups, there is only 1 event per ...
4
votes
1answer
103 views

xgboost or lightgbm to handle Binomial problems

I have a dataset containing a column of trials, a column of successes and other features; and, obviously, I can generate a probability column. I would like to use gradient boosting methods (like ...
1
vote
2answers
42 views

Time of trainig vs time of prediction, which one is used during classification algorithms comparison?

I need to use many algorithms for making a binary classification, such as Logistic regression, SVM, XGBoost, CatBoost, ... I get an interesting improvement but All of those algorithms (except LR) take ...