Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
466
questions
1
vote
0answers
10 views
Understanding additive function approximation or Understanding matching pursuit
I am trying to read Greedy function approximation: A gradient boosting machine.
On page 4 (it is marked as page 1192) under 3. Finite data the author tells how the function approximation approach ...
1
vote
1answer
11 views
why does my calibration curve for platts and isotonic have less points than my uncalibrated model?
i train a model using grid search then i use the best parameters from this to define my chosen model.
...
3
votes
1answer
44 views
feature scaling xgbRegressor
I read for example in this answer:
Does the performance of GBM methods profit from feature scaling?
that scaling doesn´t affect the performance of any tree-based method, not for lightgbm,xgboost,...
7
votes
2answers
53 views
Multi-country model or single model
I am working on a ML model to be deployed in a product operating in many countries.
The issue that I am having is the following: should I
train one model and serve it for all countries?
train a model ...
1
vote
1answer
27 views
Appropriate objective function and evaluation metric when I DO care about outliers?
I am reading these two pages:
xgboost documentation
Post on evaluation metrics
I have a dataset where I am trying to predict future spend at the user level. A lot of our spend comes from large ...
1
vote
1answer
18 views
Time series data and ML - separating training/test data
I am using XGBoost to try to predict the direction of the stock market based on social media sentiment. Having read through some studies, I was planning to separate the training/test data by time ...
0
votes
0answers
10 views
using logistic:binary as objective in xgboost
i am using 'binary:logisitic' as my objective function in xgboost classifier. Therefore when i predict probabilities i.e:
model.predict_proba(x_test)[:, 1]
will ...
1
vote
0answers
15 views
Should I run a learning curve before or after grid search? [closed]
I would like to find out how much data I should use for training my model. I also want to test what parameters I need using grid search.
My question is should I first run a learning curve on a ...
0
votes
0answers
18 views
how can i plot probability distribution of my classes in the way below?
all,
i would like to plot the following:
i have a binary classification problem where i am using xgboost as my 'model' below:
...
0
votes
1answer
34 views
Understanding output probabilites of xgboost in multiclass problems
I would like to understand the output probabilities of a xgboost classifier (or any other decision tree ensemble based classifier) in the case of a multiclass problem.
For example: We have 5 different ...
0
votes
0answers
16 views
Confidence score over xgboost logistic regression
The probabilities of logistic regression indicate how the certain the model is over predictions. if its 0.93 it means the model is 93% confident the label is 1 and 7% to be 0. or if the probability is ...
0
votes
2answers
19 views
data size requirements for XGBoost
This is a quick question. If I compare neural network and random forest, the data size requirement is huge in neural network, but a decision tree or random forest can work with less number of records ...
2
votes
1answer
18 views
XGBoost is it possible to prevent a feature from being used twice in the same tree?
I'm using XGBoost and all its doing is using the feature in the first column of my data. My feature importance chart correlates perfectly to the position of the feature in my xtrain. If I shuffle the ...
1
vote
1answer
23 views
XGBoost - feature importance just depends on the location of the feature in the data
I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data ...
1
vote
0answers
25 views
XGBoost model performance barely budging, and max_depth is basically irrelevant
I have a dataset that includes 62 features and around 1 million observations. The 62 features are mostly socioeconomic status indicators for students as they start a school year. The labels are the ...
3
votes
1answer
49 views
Why would GradientBoostClassifier do better than XGBoostClassifier? [duplicate]
I am working on the Kaggle home loan model and interestingly enough, the GradientBoostClassifier has a considerably better score than XGBClassifier. At the same time it seems to not overfit as much. (...
0
votes
1answer
32 views
XG Boost result interpretation for unbalanced datasets (Accuracy & AUCROC)
My dataset is of shape – 5621*8 (binary classification)
Label/target : Success (4324, 77 %) & Not success (1297, 23 %)
(success and Not success were been ...
1
vote
2answers
24 views
how can i compare two classification algorithms?
all,
i have two classifiers (xgboost and light gradient boosting) to predict if yes cancer or not. when i use roc_auc as my scoring method i get xgboost as 0.75 and light gradient boosting as 0.76. ...
0
votes
1answer
13 views
XGB custom objective function - small change to default regression squared error objective function
Where can I find the code for the default squared error objective function? I just want to make a small change to re-weight certain datapoints?
1
vote
1answer
13 views
How do I deal with data that has only limited target values?
I'm currently working on a small project using the D1NAMO dataset (1). I want to predict the glucose level (that is given in the dataset) based on several features: accelerometer data, heartbeat (ECG) ...
1
vote
2answers
42 views
Machine learning goal: given a population of 100,000 students, predict a group of 3,000, and minimize the median grade of that group
In other words, I am looking to predict students that will fail out of school before it happens. The data includes socioeconomic status and other related variables.
I have tried an XGB binary ...
0
votes
0answers
10 views
XGBoost parameters that will overweight errors for a specific decile
My goal is to make predictions about the bottom decile of my dataset. If I can predict the bottom decile values well (ie. minimize the errors there) then I am not worried about making accurate ...
1
vote
2answers
29 views
How to decide on using xgboost with imputation or without it and keeping missing values?
I have a large genetic dataset that I am using xgboost on to score most likely disease causing genes - giving the genes a score between 0-1 of likelihood.
I try to avoid features with a lot of ...
0
votes
1answer
28 views
XGBoost Log Loss different from GridSearchCV Log Loss
I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables ...
2
votes
2answers
23 views
Boosted tree regression loss function when data has occasionally very large values to predict?
I have a regression problem where most of my target variables are down in the range 5-30, but occasionally the target variable will spike up to 100, 500, or even 5000. These values are not spurious ...
0
votes
0answers
6 views
Zero-inflated independent feature in tree-based models
What is the best approach to include a zero-inflated continuous independent feature (e.g., 90% of the values are Zero, 10% are >0) in a Tree-based models (DT, random forest, gradient boosting. etc). I ...
0
votes
0answers
27 views
XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?
We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
1
vote
2answers
45 views
Main options on how to deal with imbalanced data
As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets:
Option 1:
Create k-fold Cross-Validation samples randomly (or even better create k-fold ...
0
votes
0answers
11 views
Building Uplift Tree using boosting
I want to build an Uplift Model for multiple treatments. To get a good model, I would like to use boosting. How is it possible to use boosting with uplift modeling although we can't really calculate ...
2
votes
2answers
29 views
Which loss function is the best loss function when using XGB regression with highly skewed dataset?
Which loss function is the best loss function when using XGB regression with a highly skewed dataset?
The skewness of the data is very high. I used XGBoost with objective function of linear ...
1
vote
1answer
32 views
XGboost and regularization
Does the XGBClassifier method utilizes the two regularization terms reg_alpha and reg_lambda,...
1
vote
0answers
13 views
testing statistical significance when comparing regression models
I have 3 models, a random forest, a XGBoost and a baseline regression model. I've performed a 5-fold cross validation on all models and use MAE as the scoring metric. So this means I basically have 3 ...
0
votes
0answers
11 views
How Xgboost or Decision trees deal with discontinous data in tabular form for time series
I am participating in a kaggle competition by the name "predict future sales". There the objective is to predict sales for each shop for each item for next month. The data is given such that only ...
0
votes
1answer
17 views
Determine how each feature contribute to XGBoost Classification
so for a summary of what I have done:
My dataset has 5 classes and 10 parameters. I used XGBclassifer from sklearn to investigate if I could use those 10 parameters to predict the class of each data ...
0
votes
0answers
12 views
Gamma objective function XGBoost
I am using XGBoost to predict a variable that is highly skewed and always is greater than zero. I did a significant search to see some materials for gamma objective function in XGBoost but I could not ...
0
votes
1answer
29 views
how does xgboost handle inf or -inf values?
all,
i am using xgboost for binary classfication. I have infs and -infs in my data due to the fact i am calcaulting ratios from one col and and another e.g. df[col1]/df[col2] , since i have zeros ...
-1
votes
1answer
29 views
Why when my local cv of loss decreases, my leaderboard's loss increases?
I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514.
I further did feature engineering and added some features to the ...
0
votes
0answers
54 views
How to implement custom loss function that has more parameters with XGBClassifier in scikit-learn?
I have following problem with implementing custom loss function with scikit-learn:
I would like to implement Focal Loss as my objective function in XGBClassifier. However, I dont know how to pass ...
0
votes
0answers
10 views
News Recommendation Engine with XGBoost
I want to build a news recommendation engine with XGBoost, but the data I have contains implicit user ratings, view history of a user. I know what my X's will(user embeddings + Item Embeddings) be but ...
1
vote
1answer
55 views
my xgboost model accuracy decreases after grid search with
I tried grid search for hyperparameter tuning in XGBoost classifier but the best accuracy is less than the accuracy without any tuning
...
1
vote
0answers
16 views
Tuning parameters for gradient boosting/xgboost
In practice, which parameter do you typically tune first? Do you tune the learning rate (or step size) first? and then tune the total number of iterations? And how do you go about tuning these ...
6
votes
2answers
376 views
What is the proper way to use early stopping with cross-validation?
I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation ...
1
vote
0answers
13 views
Is it necessary to transform data to normal distribution when removing outliers for xgboost?
sorry if this is statistics 101 but i cannot find a similar question. I am wanting to use xgboost to classify my data in two classifications. my data is numerical (financial statement data) and i can ...
1
vote
2answers
26 views
Is this over-fitting or something else?
I recently put together an entry for the House Prices Kaggle competition for beginners. I decided to try my hand at understanding and using XGBoost.
I split Kaggle's 'training' data into 'training' ...
0
votes
2answers
15 views
Getting lower performance metrics when using GridSearchCV
I have defined an XGBoost model and would like to tune some of its hyperparameters.
I am using GridSearchCV to find the best params. However, I also tried to fit the model on the entire training ...
1
vote
1answer
39 views
What's the best way to predict weekly selling data?
I am trying to create a model to predict the units that will be sold for different grocery items say in the next week.
I am structuring the problem in a three-step procedure.
Group together the ...
0
votes
0answers
11 views
expected y_pred to have dimension (1054,2) but got array with dimension (1054,)
I've used XGBoost algorithm here. The code is given below.
...
0
votes
0answers
12 views
Sagemaker - XGBOOST rank:ndcg
Does anyone know if the rank:ndcg is available on AWS Sagemaker? I am currently trying to run a model, but it seems like it's not implemented. Am I using an older xgboost version? Kinda new to AWS ...
2
votes
1answer
37 views
What is the intuitive meaning of “leaf weight” in xgboost
I looked through Tianqi Chen's presentation, but I'm struggling to understand the details of what the leaf weights are, and I would appreciate if someone could help clarify my understanding.
To put ...
0
votes
0answers
19 views
Predict best score on unlabelled test set
Data
I have one dataset with $1500$ data points, each with $\sim 23 000$ features (gene expression data, if that matters). However, I've split this dataset into a labelled training set of size 1000, ...
