Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
1
vote
0answers
10 views

Understanding additive function approximation or Understanding matching pursuit

I am trying to read Greedy function approximation: A gradient boosting machine. On page 4 (it is marked as page 1192) under 3. Finite data the author tells how the function approximation approach ...
1
vote
1answer
11 views

why does my calibration curve for platts and isotonic have less points than my uncalibrated model?

i train a model using grid search then i use the best parameters from this to define my chosen model. ...
3
votes
1answer
44 views

feature scaling xgbRegressor

I read for example in this answer: Does the performance of GBM methods profit from feature scaling? that scaling doesn´t affect the performance of any tree-based method, not for lightgbm,xgboost,...
7
votes
2answers
53 views

Multi-country model or single model

I am working on a ML model to be deployed in a product operating in many countries. The issue that I am having is the following: should I train one model and serve it for all countries? train a model ...
1
vote
1answer
27 views

Appropriate objective function and evaluation metric when I DO care about outliers?

I am reading these two pages: xgboost documentation Post on evaluation metrics I have a dataset where I am trying to predict future spend at the user level. A lot of our spend comes from large ...
1
vote
1answer
18 views

Time series data and ML - separating training/test data

I am using XGBoost to try to predict the direction of the stock market based on social media sentiment. Having read through some studies, I was planning to separate the training/test data by time ...
0
votes
0answers
10 views

using logistic:binary as objective in xgboost

i am using 'binary:logisitic' as my objective function in xgboost classifier. Therefore when i predict probabilities i.e: model.predict_proba(x_test)[:, 1] will ...
1
vote
0answers
15 views

Should I run a learning curve before or after grid search? [closed]

I would like to find out how much data I should use for training my model. I also want to test what parameters I need using grid search. My question is should I first run a learning curve on a ...
0
votes
0answers
18 views

how can i plot probability distribution of my classes in the way below?

all, i would like to plot the following: i have a binary classification problem where i am using xgboost as my 'model' below: ...
0
votes
1answer
34 views

Understanding output probabilites of xgboost in multiclass problems

I would like to understand the output probabilities of a xgboost classifier (or any other decision tree ensemble based classifier) in the case of a multiclass problem. For example: We have 5 different ...
0
votes
0answers
16 views

Confidence score over xgboost logistic regression

The probabilities of logistic regression indicate how the certain the model is over predictions. if its 0.93 it means the model is 93% confident the label is 1 and 7% to be 0. or if the probability is ...
0
votes
2answers
19 views

data size requirements for XGBoost

This is a quick question. If I compare neural network and random forest, the data size requirement is huge in neural network, but a decision tree or random forest can work with less number of records ...
2
votes
1answer
18 views

XGBoost is it possible to prevent a feature from being used twice in the same tree?

I'm using XGBoost and all its doing is using the feature in the first column of my data. My feature importance chart correlates perfectly to the position of the feature in my xtrain. If I shuffle the ...
1
vote
1answer
23 views

XGBoost - feature importance just depends on the location of the feature in the data

I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data ...
1
vote
0answers
25 views

XGBoost model performance barely budging, and max_depth is basically irrelevant

I have a dataset that includes 62 features and around 1 million observations. The 62 features are mostly socioeconomic status indicators for students as they start a school year. The labels are the ...
3
votes
1answer
49 views

Why would GradientBoostClassifier do better than XGBoostClassifier? [duplicate]

I am working on the Kaggle home loan model and interestingly enough, the GradientBoostClassifier has a considerably better score than XGBClassifier. At the same time it seems to not overfit as much. (...
0
votes
1answer
32 views

XG Boost result interpretation for unbalanced datasets (Accuracy & AUCROC)

My dataset is of shape – 5621*8 (binary classification) Label/target : Success (4324, 77 %) & Not success (1297, 23 %) (success and Not success were been ...
1
vote
2answers
24 views

how can i compare two classification algorithms?

all, i have two classifiers (xgboost and light gradient boosting) to predict if yes cancer or not. when i use roc_auc as my scoring method i get xgboost as 0.75 and light gradient boosting as 0.76. ...
0
votes
1answer
13 views

XGB custom objective function - small change to default regression squared error objective function

Where can I find the code for the default squared error objective function? I just want to make a small change to re-weight certain datapoints?
1
vote
1answer
13 views

How do I deal with data that has only limited target values?

I'm currently working on a small project using the D1NAMO dataset (1). I want to predict the glucose level (that is given in the dataset) based on several features: accelerometer data, heartbeat (ECG) ...
1
vote
2answers
42 views

Machine learning goal: given a population of 100,000 students, predict a group of 3,000, and minimize the median grade of that group

In other words, I am looking to predict students that will fail out of school before it happens. The data includes socioeconomic status and other related variables. I have tried an XGB binary ...
0
votes
0answers
10 views

XGBoost parameters that will overweight errors for a specific decile

My goal is to make predictions about the bottom decile of my dataset. If I can predict the bottom decile values well (ie. minimize the errors there) then I am not worried about making accurate ...
1
vote
2answers
29 views

How to decide on using xgboost with imputation or without it and keeping missing values?

I have a large genetic dataset that I am using xgboost on to score most likely disease causing genes - giving the genes a score between 0-1 of likelihood. I try to avoid features with a lot of ...
0
votes
1answer
28 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables ...
2
votes
2answers
23 views

Boosted tree regression loss function when data has occasionally very large values to predict?

I have a regression problem where most of my target variables are down in the range 5-30, but occasionally the target variable will spike up to 100, 500, or even 5000. These values are not spurious ...
0
votes
0answers
6 views

Zero-inflated independent feature in tree-based models

What is the best approach to include a zero-inflated continuous independent feature (e.g., 90% of the values are Zero, 10% are >0) in a Tree-based models (DT, random forest, gradient boosting. etc). I ...
0
votes
0answers
27 views

XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?

We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
1
vote
2answers
45 views

Main options on how to deal with imbalanced data

As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets: Option 1: Create k-fold Cross-Validation samples randomly (or even better create k-fold ...
0
votes
0answers
11 views

Building Uplift Tree using boosting

I want to build an Uplift Model for multiple treatments. To get a good model, I would like to use boosting. How is it possible to use boosting with uplift modeling although we can't really calculate ...
2
votes
2answers
29 views

Which loss function is the best loss function when using XGB regression with highly skewed dataset?

Which loss function is the best loss function when using XGB regression with a highly skewed dataset? The skewness of the data is very high. I used XGBoost with objective function of linear ...
1
vote
1answer
32 views

XGboost and regularization

Does the XGBClassifier method utilizes the two regularization terms reg_alpha and reg_lambda,...
1
vote
0answers
13 views

testing statistical significance when comparing regression models

I have 3 models, a random forest, a XGBoost and a baseline regression model. I've performed a 5-fold cross validation on all models and use MAE as the scoring metric. So this means I basically have 3 ...
0
votes
0answers
11 views

How Xgboost or Decision trees deal with discontinous data in tabular form for time series

I am participating in a kaggle competition by the name "predict future sales". There the objective is to predict sales for each shop for each item for next month. The data is given such that only ...
0
votes
1answer
17 views

Determine how each feature contribute to XGBoost Classification

so for a summary of what I have done: My dataset has 5 classes and 10 parameters. I used XGBclassifer from sklearn to investigate if I could use those 10 parameters to predict the class of each data ...
0
votes
0answers
12 views

Gamma objective function XGBoost

I am using XGBoost to predict a variable that is highly skewed and always is greater than zero. I did a significant search to see some materials for gamma objective function in XGBoost but I could not ...
0
votes
1answer
29 views

how does xgboost handle inf or -inf values?

all, i am using xgboost for binary classfication. I have infs and -infs in my data due to the fact i am calcaulting ratios from one col and and another e.g. df[col1]/df[col2] , since i have zeros ...
-1
votes
1answer
29 views

Why when my local cv of loss decreases, my leaderboard's loss increases?

I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514. I further did feature engineering and added some features to the ...
0
votes
0answers
54 views

How to implement custom loss function that has more parameters with XGBClassifier in scikit-learn?

I have following problem with implementing custom loss function with scikit-learn: I would like to implement Focal Loss as my objective function in XGBClassifier. However, I dont know how to pass ...
0
votes
0answers
10 views

News Recommendation Engine with XGBoost

I want to build a news recommendation engine with XGBoost, but the data I have contains implicit user ratings, view history of a user. I know what my X's will(user embeddings + Item Embeddings) be but ...
1
vote
1answer
55 views

my xgboost model accuracy decreases after grid search with

I tried grid search for hyperparameter tuning in XGBoost classifier but the best accuracy is less than the accuracy without any tuning ...
1
vote
0answers
16 views

Tuning parameters for gradient boosting/xgboost

In practice, which parameter do you typically tune first? Do you tune the learning rate (or step size) first? and then tune the total number of iterations? And how do you go about tuning these ...
6
votes
2answers
376 views

What is the proper way to use early stopping with cross-validation?

I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
1
vote
0answers
13 views

Is it necessary to transform data to normal distribution when removing outliers for xgboost?

sorry if this is statistics 101 but i cannot find a similar question. I am wanting to use xgboost to classify my data in two classifications. my data is numerical (financial statement data) and i can ...
1
vote
2answers
26 views

Is this over-fitting or something else?

I recently put together an entry for the House Prices Kaggle competition for beginners. I decided to try my hand at understanding and using XGBoost. I split Kaggle's 'training' data into 'training' ...
0
votes
2answers
15 views

Getting lower performance metrics when using GridSearchCV

I have defined an XGBoost model and would like to tune some of its hyperparameters. I am using GridSearchCV to find the best params. However, I also tried to fit the model on the entire training ...
1
vote
1answer
39 views

What's the best way to predict weekly selling data?

I am trying to create a model to predict the units that will be sold for different grocery items say in the next week. I am structuring the problem in a three-step procedure. Group together the ...
0
votes
0answers
11 views

expected y_pred to have dimension (1054,2) but got array with dimension (1054,)

I've used XGBoost algorithm here. The code is given below. ...
0
votes
0answers
12 views

Sagemaker - XGBOOST rank:ndcg

Does anyone know if the rank:ndcg is available on AWS Sagemaker? I am currently trying to run a model, but it seems like it's not implemented. Am I using an older xgboost version? Kinda new to AWS ...
2
votes
1answer
37 views

What is the intuitive meaning of “leaf weight” in xgboost

I looked through Tianqi Chen's presentation, but I'm struggling to understand the details of what the leaf weights are, and I would appreciate if someone could help clarify my understanding. To put ...
0
votes
0answers
19 views

Predict best score on unlabelled test set

Data I have one dataset with $1500$ data points, each with $\sim 23 000$ features (gene expression data, if that matters). However, I've split this dataset into a labelled training set of size 1000, ...

1
2 3 4 5
10