Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
681
questions
0
votes
0
answers
18
views
How to optimize hyperparameters of Xgboost algorithm in multiple classes in R?
My unbalanced dataset with 4 classes. there is. I want to use xgboost algorithm for classification. But despite my research I couldn't find how to optimize hyperparameters for multiclass datasets. ...
- 1
0
votes
0
answers
39
views
Time series Classifiction - is LSTM better than XGBoost
My question is whether LSTM RNN is a better predictor of an label (note not forecaster) than XGBoost. Thus far I have had moderate success with XGB but I wonder if the tree nature, random start and ...
- 101
0
votes
0
answers
19
views
Objective function in optuna seems to return at random points
I am trying to use a K-fold cross-validation within the optuna objective function.
Unfortunately, the output of the objective function seems to pop out at random places, not at the end of the cross-...
0
votes
1
answer
22
views
What do I make of all classification scores being equal to 1?
I've built an XGBoost classifier on a dataset that has 51 columns and a 1000 rows with following code:
...
- 103
0
votes
0
answers
36
views
Do we need to check the training score when we use randomizedsearchcv
given a model and a set of parameters, randomizedsearchCV(or gridsearchCV) gives the mean of the best scores from a list of different folds of the datasets.
Does the model control for overfitting? I ...
- 21
0
votes
1
answer
35
views
how do I test if overfitting exists when I use cross_val_score method?
I got the following code form a book on xgboost.
I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes.
mean accuracy is 81 which can be okay. but what if ...
- 21
1
vote
0
answers
13
views
XGBoost learning rate "rescaling according to size of trees". Explanation
I'm trying to understand the impact of the learning rate parameter in XGBoost. I started inspecting the source code. In this file, I found the following lines
...
0
votes
1
answer
23
views
6
votes
1
answer
84
views
How to choose between different models with similar results? RF, GLM and XGBoost
I am a medical doctor trying to make prediction models based on a database of approximately 1500 patients with 60+ parameters each.
I am dealing with a classification problem (mortality at 1, 3, 6 and ...
- 61
0
votes
0
answers
4
views
How to give a column a special weight for any algorithm? (binary option)
I'm dealing with binary option, I'm making a classification model using xgboost, my idea is to predict the posterior candle color by using some data of the 20 previous candles so I made a dataset on ...
1
vote
0
answers
61
views
ValueError: Found input variables with inconsistent numbers of samples: [120, 30]
I practice XGBClassifier() to predict the target in iris dataset.
here is the code:
...
0
votes
1
answer
52
views
Machine learning | Column names vs number when training/predicting
Been doing machine learning since a few months by now. I've a grounding questions that I couldn't answer by my self. It's possible I'm asking the wrong question:
When training models, like XGBoost, ...
- 3
0
votes
0
answers
13
views
How to Forecast Stock Index against Global Indicators
My goal is predict the benchmark index of the Indian stock market with the help machine learning algorithm using of global market indices as mentioned below. Put simply, forecast whether tomorrow’s ...
0
votes
0
answers
19
views
time series predict into future
I am using Python and stats models SARIMA and I can predict out 24 samples like shown below.
Can I do the same thing with machine learning but (not LSTM) and not have to feed an array of data through ...
- 381
0
votes
1
answer
84
views
Feature importance in xgboost
I've been reading that feature importance in xgboost is computed the same way as in random forests. However, the learning rate reduces the effect of downstream trees. Is the learning rate taken into ...
- 65
0
votes
1
answer
27
views
How can I use XGBoost in SciKit-Learn on an Air-gapped Computer?
I'd like to use XGBoost in SciKit-Learn, however, I'm on an Air-Gapped computer and can't install it normally using pip.
How can I install XGBoost on an air-gapped computer?
- 511
0
votes
0
answers
17
views
Multi-class classification model unable to return desired outcome
I have a scenario of multi-class dataset with around 10 distinct classes of target. There are 3 categorical features each with multiple labels. If we check the data, each unique combination of feature ...
0
votes
0
answers
79
views
How to use early_stopping_rounds in the Final Model? (CatBoost example with Optuna)
Imagine we have a model in the sklearn pipeline:
...
- 1
0
votes
1
answer
30
views
Is there any benefit to using cross validation from the XGBoost library over sklearn when tuning hyperparameters?
The XGBoost library has its own implementation of cross validation through xgboost.cv(). It looks like it requires data be stored as a DMatrix.
Instead of using <...
- 101
1
vote
1
answer
21
views
How to prevent my model from mistaking categorical feature for ordinal feature
I have tabular data where each group of 100 rows represents a deployment of a specific geometry that has certain features measured. For example, I have 10000 deployments stored in a column called &...
0
votes
2
answers
551
views
Shap summary Plot for binary classification and multiclass
For binary classification, I am getting only a unicolor feature importance plot (i.e., the two classes do not appear individually).
However, for multiclass, I am getting feature importance in ...
- 1
0
votes
0
answers
24
views
Understanding loss function/ implementing my own
I am currently working on an ETA prediction LightGBM model (regression tree) for which I want the negative residuals to be penalized higher than the positive. I understand that a custom loss function ...
0
votes
1
answer
22
views
Need my xgboost model to be more liberal with classifications
I have an xgboost model that predicts the likelihood of a sales lead to close (actually to turn into an "opportunity" which is one step before the close but that's beside the point). The ...
- 103
0
votes
0
answers
41
views
Best way to handle missing values with XGBoost?
I know of a number of ways to handle missing feature values, and wanted to get folks' input on what might work best. My end goal is to be able to predict accurate probabilities for a binary ...
- 101
0
votes
0
answers
40
views
Implementing XGBoost with CNN
I am trying to implement XGBoost as a classifier for a pre-trained CNN. The model produces an F1 score of 93, however, when classifying with XGBoost (or with SVM), the F1 drops to 33. It seems to be ...
- 1
0
votes
0
answers
19
views
Interpreting high test accuracy
I am trying to build a classification xgboost model at work, and I'm potentially facing overfitting issue that I have never seen before.
My training sample size is 320,000 X 718 and testing sample is ...
- 121
0
votes
3
answers
172
views
Unable to build a XGBoost classifier that gives good precision and recall on highly imbalanced data
The XGBoost Classifier I built is consistently returning a f1 score of 0 and I am unable to fix this despite experimenting with various hyperparameters. The data is heavily imbalanced and hence I feel ...
- 13
0
votes
0
answers
36
views
How to interpret the Shapley values returned by TreeShap for multiclass classification?
I have read in Molnar (2022) and Gianfagna (2021) books that the TreeShap method returns the exact Shapley values of Shapley (1951).
The Shapley value estimates, given the current set of feature ...
- 126
0
votes
0
answers
45
views
Forecasting multiple univariate time series using lagged values as predictors
I'm trying to forecast 20 univariate time series using an ensemble of XGBoost, Prophet, Prophet Boost and Random Forests.
As you can imagine, each individual time series have statistically significant ...
- 261
3
votes
1
answer
113
views
Optuna Median Pruner n_warmup_steps
For Gradient Boosting Models such as XGBOOST and LGBM does n_warmup_steps in optuna.pruners.MedianPruner refer to the minimum number of folds evaluated before pruning is triggered?
I.e. if number of ...
- 131
0
votes
0
answers
55
views
XGBoost Gradient Boosting and Gradient Descent confusion
I am trying to understand how the XGBoost determines the next tree. Some sources state that the model uses gradient descent to find the optimal option:
This answer from a question on this s.e. also ...
- 1
4
votes
1
answer
258
views
What does it mean when a model is more conservative?
I'm trying to tune some parameters in XGBoost and read a lot about "...makes to model more conservative". Can somebody explain me what the word conservative means in this case?
I can imagine ...
- 143
1
vote
1
answer
26
views
Why does feature selection matter if your model has L1 regularization?
I've been tinkering around with boosted trees, and I saw that for common libraries there is a parameter you can set to determine L1 regularization. I doubled my original feature set to around 130 ...
- 11
0
votes
0
answers
123
views
How is train test split in xgboost cv specified?
It is to be noted that the xgboost.cv method returns eval metrics on both train and test sets whereas the function itself takes no parameter stating which dataset to be used for training and which for ...
- 73
0
votes
0
answers
16
views
compare decision tree vs extended gradient boosting mathematically?
If we want to compare decision tree vs extended gradient boosting vs xgboost mathematically, what are their differences?
- 1,754
0
votes
2
answers
40
views
Can Gradient Descent boosted decision trees miss the forest for the trees?
My understanding of this stuff is pretty basic so my semantics may be off, but bare with me.
XGBoost and other gradient descent packages make the best possible split of the data right off the bat. ...
- 113
0
votes
0
answers
30
views
Feature Importance and Threshold Moving
Problem Type : Binary Classification
Dataset : Imbalanced
Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9.
Is it viable ...
- 1
0
votes
1
answer
30
views
Why compare multiple machine learning algorithms and then decide which algorithm to use for fine tuning?
I have a problem.
There is a dataset A, which deals with a classification problem. And for this dataset, several different baseline algorithms have been defined and computed.
In addition, three models ...
- 89
0
votes
2
answers
103
views
xgboost reduce float precision does not reduce train time
I expected that reducing the precision of my data (e.g., from int64 to int8) would speed up the training.
But, even if I reduce the overall size of my dataset by 74%, I do not see an improvement.
Is ...
- 111
0
votes
0
answers
38
views
XGBoost: Why does model.feature_importances_ return zero values?
I used XGBClassifier to train a model on numerical datasets. Hyperparameters are as follows:
...
- 101
0
votes
0
answers
37
views
is time series data normalization for xgboost required?
According to the developer - xgboost does not require feature normalization https://github.com/dmlc/xgboost/issues/357
no you do not have to normalize the ...
- 101
0
votes
1
answer
28
views
Training XGBoost on time series features of varying sample length
I have some time series data that contain features that that go back anywhere from 5 to 50 years. I've considered imputation (e.g. taking the mean), but I'm not sure it's feasible to impute such large ...
- 11
1
vote
0
answers
101
views
How can I write a custom loss function to punish lower predicted values?
I am trying to write a custom loss function for XGBRegressor that needs to punish predicted values that are under some arbitrary threshold. The code I came up with does not affect the results at all, ...
- 11
0
votes
1
answer
104
views
XGBoost training on sample of time series data
I am new to XGBoost and would like to use it on a time series dataset. Here is the scenario I'm faced with:
The data set contains N samples of length T, with N>>T. I'd like to train an XGBoost ...
- 101
4
votes
0
answers
54
views
Does ROC AUC different between crossval and test set indicate overfitting or other problem?
I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
- 116
1
vote
1
answer
213
views
Online Learning/Continual Learning for tree-based Algorithms
Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
- 318
0
votes
0
answers
7
views
Part random sample, part force sample with multiple event=0
Question: Is this approach sound?
Goal: Model probability of event = 1
Problem: Time series with multiple event = 0 per id, only one event=1
Approach:
Get all records where event = 1 (not random ...
- 1
0
votes
1
answer
53
views
combine xgb feature importance
Is it possible to combine the results from the xgb.boost importance function. For example, due to one hot encoding, I have a feature age=35 and another age=60. Is there a way that I can add these to ...
1
vote
1
answer
155
views
scale_pos_weight effect in XGBClassifier
I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier.
It says everywhere to set it to Count of negatives / Count of ...
- 171
0
votes
0
answers
93
views
XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression
i'm tring tune XGBoost model for my dataset.
...
