Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

How to optimize hyperparameters of Xgboost algorithm in multiple classes in R?

My unbalanced dataset with 4 classes. there is. I want to use xgboost algorithm for classification. But despite my research I couldn't find how to optimize hyperparameters for multiclass datasets. ...
  • 1
0 votes
0 answers
39 views

Time series Classifiction - is LSTM better than XGBoost

My question is whether LSTM RNN is a better predictor of an label (note not forecaster) than XGBoost. Thus far I have had moderate success with XGB but I wonder if the tree nature, random start and ...
0 votes
0 answers
19 views

Objective function in optuna seems to return at random points

I am trying to use a K-fold cross-validation within the optuna objective function. Unfortunately, the output of the objective function seems to pop out at random places, not at the end of the cross-...
0 votes
1 answer
22 views

What do I make of all classification scores being equal to 1?

I've built an XGBoost classifier on a dataset that has 51 columns and a 1000 rows with following code: ...
  • 103
0 votes
0 answers
36 views

Do we need to check the training score when we use randomizedsearchcv

given a model and a set of parameters, randomizedsearchCV(or gridsearchCV) gives the mean of the best scores from a list of different folds of the datasets. Does the model control for overfitting? I ...
0 votes
1 answer
35 views

how do I test if overfitting exists when I use cross_val_score method?

I got the following code form a book on xgboost. I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes. mean accuracy is 81 which can be okay. but what if ...
1 vote
0 answers
13 views

XGBoost learning rate "rescaling according to size of trees". Explanation

I'm trying to understand the impact of the learning rate parameter in XGBoost. I started inspecting the source code. In this file, I found the following lines ...
0 votes
1 answer
23 views

XGB DataFrame.dtypes error

Here is the code to assign the variables ...
  • 1
6 votes
1 answer
84 views

How to choose between different models with similar results? RF, GLM and XGBoost

I am a medical doctor trying to make prediction models based on a database of approximately 1500 patients with 60+ parameters each. I am dealing with a classification problem (mortality at 1, 3, 6 and ...
0 votes
0 answers
4 views

How to give a column a special weight for any algorithm? (binary option)

I'm dealing with binary option, I'm making a classification model using xgboost, my idea is to predict the posterior candle color by using some data of the 20 previous candles so I made a dataset on ...
1 vote
0 answers
61 views

ValueError: Found input variables with inconsistent numbers of samples: [120, 30]

I practice XGBClassifier() to predict the target in iris dataset. here is the code: ...
0 votes
1 answer
52 views

Machine learning | Column names vs number when training/predicting

Been doing machine learning since a few months by now. I've a grounding questions that I couldn't answer by my self. It's possible I'm asking the wrong question: When training models, like XGBoost, ...
0 votes
0 answers
13 views

How to Forecast Stock Index against Global Indicators

My goal is predict the benchmark index of the Indian stock market with the help machine learning algorithm using of global market indices as mentioned below. Put simply, forecast whether tomorrow’s ...
0 votes
0 answers
19 views

time series predict into future

I am using Python and stats models SARIMA and I can predict out 24 samples like shown below. Can I do the same thing with machine learning but (not LSTM) and not have to feed an array of data through ...
  • 381
0 votes
1 answer
84 views

Feature importance in xgboost

I've been reading that feature importance in xgboost is computed the same way as in random forests. However, the learning rate reduces the effect of downstream trees. Is the learning rate taken into ...
  • 65
0 votes
1 answer
27 views

How can I use XGBoost in SciKit-Learn on an Air-gapped Computer?

I'd like to use XGBoost in SciKit-Learn, however, I'm on an Air-Gapped computer and can't install it normally using pip. How can I install XGBoost on an air-gapped computer?
  • 511
0 votes
0 answers
17 views

Multi-class classification model unable to return desired outcome

I have a scenario of multi-class dataset with around 10 distinct classes of target. There are 3 categorical features each with multiple labels. If we check the data, each unique combination of feature ...
0 votes
0 answers
79 views

How to use early_stopping_rounds in the Final Model? (CatBoost example with Optuna)

Imagine we have a model in the sklearn pipeline: ...
0 votes
1 answer
30 views

Is there any benefit to using cross validation from the XGBoost library over sklearn when tuning hyperparameters?

The XGBoost library has its own implementation of cross validation through xgboost.cv(). It looks like it requires data be stored as a DMatrix. Instead of using <...
  • 101
1 vote
1 answer
21 views

How to prevent my model from mistaking categorical feature for ordinal feature

I have tabular data where each group of 100 rows represents a deployment of a specific geometry that has certain features measured. For example, I have 10000 deployments stored in a column called &...
0 votes
2 answers
551 views

Shap summary Plot for binary classification and multiclass

For binary classification, I am getting only a unicolor feature importance plot (i.e., the two classes do not appear individually). However, for multiclass, I am getting feature importance in ...
  • 1
0 votes
0 answers
24 views

Understanding loss function/ implementing my own

I am currently working on an ETA prediction LightGBM model (regression tree) for which I want the negative residuals to be penalized higher than the positive. I understand that a custom loss function ...
0 votes
1 answer
22 views

Need my xgboost model to be more liberal with classifications

I have an xgboost model that predicts the likelihood of a sales lead to close (actually to turn into an "opportunity" which is one step before the close but that's beside the point). The ...
0 votes
0 answers
41 views

Best way to handle missing values with XGBoost?

I know of a number of ways to handle missing feature values, and wanted to get folks' input on what might work best. My end goal is to be able to predict accurate probabilities for a binary ...
  • 101
0 votes
0 answers
40 views

Implementing XGBoost with CNN

I am trying to implement XGBoost as a classifier for a pre-trained CNN. The model produces an F1 score of 93, however, when classifying with XGBoost (or with SVM), the F1 drops to 33. It seems to be ...
  • 1
0 votes
0 answers
19 views

Interpreting high test accuracy

I am trying to build a classification xgboost model at work, and I'm potentially facing overfitting issue that I have never seen before. My training sample size is 320,000 X 718 and testing sample is ...
0 votes
3 answers
172 views

Unable to build a XGBoost classifier that gives good precision and recall on highly imbalanced data

The XGBoost Classifier I built is consistently returning a f1 score of 0 and I am unable to fix this despite experimenting with various hyperparameters. The data is heavily imbalanced and hence I feel ...
0 votes
0 answers
36 views

How to interpret the Shapley values returned by TreeShap for multiclass classification?

I have read in Molnar (2022) and Gianfagna (2021) books that the TreeShap method returns the exact Shapley values of Shapley (1951). The Shapley value estimates, given the current set of feature ...
  • 126
0 votes
0 answers
45 views

Forecasting multiple univariate time series using lagged values as predictors

I'm trying to forecast 20 univariate time series using an ensemble of XGBoost, Prophet, Prophet Boost and Random Forests. As you can imagine, each individual time series have statistically significant ...
  • 261
3 votes
1 answer
113 views

Optuna Median Pruner n_warmup_steps

For Gradient Boosting Models such as XGBOOST and LGBM does n_warmup_steps in optuna.pruners.MedianPruner refer to the minimum number of folds evaluated before pruning is triggered? I.e. if number of ...
0 votes
0 answers
55 views

XGBoost Gradient Boosting and Gradient Descent confusion

I am trying to understand how the XGBoost determines the next tree. Some sources state that the model uses gradient descent to find the optimal option: This answer from a question on this s.e. also ...
4 votes
1 answer
258 views

What does it mean when a model is more conservative?

I'm trying to tune some parameters in XGBoost and read a lot about "...makes to model more conservative". Can somebody explain me what the word conservative means in this case? I can imagine ...
  • 143
1 vote
1 answer
26 views

Why does feature selection matter if your model has L1 regularization?

I've been tinkering around with boosted trees, and I saw that for common libraries there is a parameter you can set to determine L1 regularization. I doubled my original feature set to around 130 ...
0 votes
0 answers
123 views

How is train test split in xgboost cv specified?

It is to be noted that the xgboost.cv method returns eval metrics on both train and test sets whereas the function itself takes no parameter stating which dataset to be used for training and which for ...
  • 73
0 votes
0 answers
16 views

compare decision tree vs extended gradient boosting mathematically?

If we want to compare decision tree vs extended gradient boosting vs xgboost mathematically, what are their differences?
  • 1,754
0 votes
2 answers
40 views

Can Gradient Descent boosted decision trees miss the forest for the trees?

My understanding of this stuff is pretty basic so my semantics may be off, but bare with me. XGBoost and other gradient descent packages make the best possible split of the data right off the bat. ...
0 votes
0 answers
30 views

Feature Importance and Threshold Moving

Problem Type : Binary Classification Dataset : Imbalanced Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9. Is it viable ...
  • 1
0 votes
1 answer
30 views

Why compare multiple machine learning algorithms and then decide which algorithm to use for fine tuning?

I have a problem. There is a dataset A, which deals with a classification problem. And for this dataset, several different baseline algorithms have been defined and computed. In addition, three models ...
  • 89
0 votes
2 answers
103 views

xgboost reduce float precision does not reduce train time

I expected that reducing the precision of my data (e.g., from int64 to int8) would speed up the training. But, even if I reduce the overall size of my dataset by 74%, I do not see an improvement. Is ...
  • 111
0 votes
0 answers
38 views

XGBoost: Why does model.feature_importances_ return zero values?

I used XGBClassifier to train a model on numerical datasets. Hyperparameters are as follows: ...
  • 101
0 votes
0 answers
37 views

is time series data normalization for xgboost required?

According to the developer - xgboost does not require feature normalization https://github.com/dmlc/xgboost/issues/357 no you do not have to normalize the ...
  • 101
0 votes
1 answer
28 views

Training XGBoost on time series features of varying sample length

I have some time series data that contain features that that go back anywhere from 5 to 50 years. I've considered imputation (e.g. taking the mean), but I'm not sure it's feasible to impute such large ...
1 vote
0 answers
101 views

How can I write a custom loss function to punish lower predicted values?

I am trying to write a custom loss function for XGBRegressor that needs to punish predicted values that are under some arbitrary threshold. The code I came up with does not affect the results at all, ...
  • 11
0 votes
1 answer
104 views

XGBoost training on sample of time series data

I am new to XGBoost and would like to use it on a time series dataset. Here is the scenario I'm faced with: The data set contains N samples of length T, with N>>T. I'd like to train an XGBoost ...
4 votes
0 answers
54 views

Does ROC AUC different between crossval and test set indicate overfitting or other problem?

I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
1 vote
1 answer
213 views

Online Learning/Continual Learning for tree-based Algorithms

Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
0 votes
0 answers
7 views

Part random sample, part force sample with multiple event=0

Question: Is this approach sound? Goal: Model probability of event = 1 Problem: Time series with multiple event = 0 per id, only one event=1 Approach: Get all records where event = 1 (not random ...
  • 1
0 votes
1 answer
53 views

combine xgb feature importance

Is it possible to combine the results from the xgb.boost importance function. For example, due to one hot encoding, I have a feature age=35 and another age=60. Is there a way that I can add these to ...
1 vote
1 answer
155 views

scale_pos_weight effect in XGBClassifier

I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier. It says everywhere to set it to Count of negatives / Count of ...
  • 171
0 votes
0 answers
93 views

XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression

i'm tring tune XGBoost model for my dataset. ...

1
2 3 4 5
14