Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
699
questions
0
votes
0
answers
11
views
How to deal with missing values in the output for XGBoost Regressor
So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value.
This is because ...
- 1
0
votes
0
answers
23
views
Custom objective function for xgboost to optimize lift in best decile
I've tried to define a custom objective function for xgboost to optimize the lift for a binary classification problem in the upper decile. The task is simply to concentrate the training effort to the ...
0
votes
1
answer
102
views
Understand and compute confidence interval and coefficient of variation for regression model
I would like to better understand the concepts of:
coefficient of variation and confidence interval.
Trivially taking the definitions from wikipedia:
confidence interval (CI)
In frequentist ...
- 1
0
votes
0
answers
23
views
Ordinal log-loss in a multiclass classification in XGBoost?
I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High).
What I would ...
- 111
0
votes
0
answers
36
views
Making ML Model (XGBoost) Output Smoother
I am using XGBoost for a classification problem. The model has multiple inputs and a probability as output.
For simplicity's sake, let's say the model only has one continous input and the function to ...
- 11
0
votes
0
answers
5
views
Histogram creation in lightgbm in the train API and the scikit-learn API. Is it always benefitial to use the train API?
In the LightGBM for python we have a scikit-learn API in which (either for regression or for classification) there is fit method whose documentation is
fit(X, y, sample_weight=None, init_score=None, ...
- 821
0
votes
0
answers
14
views
Xgboost giving different results on windows and Linux
I'm facing one issue with the results not matching on Windows and Linux. I'm using the same code on both machines. Using separate trains. csv and test. csv
The problem is due to sampling and ...
- 1
0
votes
1
answer
24
views
Validity of using raw time series data for training of xgboost/random forest classifier
I am currently working on a project aiming of classification of process states based on time series data. For this, we are looking at different models, such as XGBoost-based classifiers or ...
- 103
0
votes
0
answers
15
views
Minimize MAE loss for a target that is sum of two other targets
Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
0
votes
0
answers
21
views
Binary classification using xgboost
Why when adding new features in my ADS for a binary classification using XGBOOST my score and uplift has decreased ? What is the best way to treate categorical features or other features in order that ...
0
votes
0
answers
4
views
Parameter outside specified range using HGboost/Hyperopt library
I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
- 101
1
vote
2
answers
82
views
Xgboost model predicting extreme values for events and non-events | Overfitting
Extreme values are predicted by my trained xgboost classification model in BQML for both events (Y=1) and non-events (Y=0). For all event observations, the model calculates probability scores that ...
- 133
0
votes
0
answers
20
views
XGBoost Classifier Evaluation Confusion on New Dataset Despite High Cross-Validation Scores
I have built an XGBoost classifier model with 90 features, trained on a dataset containing 760k samples. I took great care to separate the labels from the features in both the training and testing ...
- 1
1
vote
0
answers
33
views
0
votes
0
answers
31
views
XGBoost Classifier + Isotonic Regression leading to worse probability accuracy
I'm testing out an XGBoost Classifier with the goal of using the probabilities it predicts in production. I know that tree based model probabilities are often not calibrated well so I decided to test ...
- 1
0
votes
0
answers
46
views
XGBoost tweedie regression objetive from scratch
I'm trying to gain a deeper understanding of the tweedie loss function and how it is used in XGBoost. So, I tried to implement it from scratch. I started by examining the original implementation.
I ...
0
votes
0
answers
40
views
XGBRegressor underestimates sum of regression target (insurance application)
I'm trying to learn how to apply Boostig Algorithms (e.g. XGBoost) to insurance applications (premium calculation). As a starting point I used this tutorial from the scikit-learn website.
tldr: The ...
1
vote
1
answer
50
views
Quality of first tree prediction in xgboost
Up until now I have mostly been using neural nets for regression. There on each iteration (epoch) the weights are updated using the whole training data, and we expect the error to go down each time, ...
- 111
0
votes
0
answers
44
views
Problem calculating f-1 score when when running hyperparameter tuning using Optuna
I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
- 11
1
vote
0
answers
43
views
Replication of XGBoost's binary:logistic loss
I am trying to replicate XGBoost's logistic loss function as a first step before implementing my own custom loss functions. Following from here and looking at the original code in git repository, I ...
- 111
1
vote
1
answer
51
views
Time series prediction on test set produces multiple cycles instead of following trend
I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use.
There's just one feature ...
0
votes
1
answer
60
views
XGBoost with objective as binary:logistic returning only binary value
I am training an xgboost model for binary classification using objective as 'binary:logistic'. The model should predict probability but it is outputting either 0 or 1. I want the model to output the ...
- 120
1
vote
3
answers
85
views
Feature importance score for a feature that contains mostly 0's in XGBoost
I have read that the feature importance scores are calculated based on how a split on that feature improves performance.
I have a binary classification dataset and am running XGBoost classifier on it. ...
- 11
0
votes
1
answer
45
views
eval_metric of XGBoost // ML model in general
Say I am using Xgboost on a binary classification task. eval_metric is one of the model parameter. How should I think about the impact of using different eval_metric(e.g rmse/mae/logloss) in general? ...
- 11
0
votes
0
answers
53
views
Can I decompose SHAP interaction values like a linear regression?
I had a question regarding the shap interaction matrix.
Suppose I have 500 samples with 2 features. Then my interaction matrix becomes (500,2,2).
I want to calculate the SHAP values of each feature ...
0
votes
0
answers
47
views
Build a model with cross-validation on entire dataset to learn insights?
Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed.
Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
0
votes
0
answers
54
views
XGBoost Architecture Diagram required
Good Day!
My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
- 1
0
votes
0
answers
31
views
weight derivation for XGBOOST in regression trees
I am working on a variation of the XGBOOST presented on the following paper by Chen and Guestrin: https://dl.acm.org/doi/abs/10.1145/2939672.2939785
My goal is to fit a linear regression in each leaf ...
0
votes
0
answers
41
views
How to predict an xgboost model outcome directly from the trained trees?
I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it ...
- 103
1
vote
1
answer
118
views
Funny looking learning curve
I am training an XGBoost for binary classification. I am getting a very odd-looking learning curve. Any idea what is going on and how I might fix this? Thanks
- 13
1
vote
1
answer
131
views
Why is accuracy score suddenly becoming 1 on using XGBoost?
I am developing a music classification system based on a kaggle dataset: https://www.kaggle.com/datasets/vatsalmavani/spotify-dataset
I tried using K means classifier to classify the songs into 4 ...
0
votes
0
answers
36
views
Why am I getting differing "gain" feature importances from XGBoost?
I am currently training an XGBoost model for binary classification. I have fitted and predicted with the model but when I try to get the "gain" type feature importances, the results differ ...
0
votes
0
answers
116
views
How do the predict() and predict_proba() for xgbClassifier() work?
I am using xgboost classifier for a binary classification problem. I used the predict() to get the class predictions (0/1) but I also used the predict_proba() method to get the class probabilities (my ...
- 1,091
0
votes
1
answer
221
views
How to intrepret low F1 score and high AUC on training set?
I am currently working on a very imbalanced dataset:
24 million transactions (rows of data)
30,000 fraudulent transactions
(0.1% of total transactions)
and I am using XGBoost as the model to predict ...
0
votes
0
answers
47
views
Why does my accuracy score drop after hyperparameter tuning in XGBoost?
I am trying to tune the model I've built, but every time I change hyperparameters my accuracy score drops significantly. I'm using RandomizedSearchCV and best_params_ to determine which parameters I ...
0
votes
0
answers
76
views
Visualize Catboost and XGBoost training process + Cross Validation
I want to optimize Catboost and XGBoost models and visualize this process such that:
Use 3-fold cross-validation
Use my own pre-processing pipeline (Missing value imputation, over- or undersampling)
...
- 61
0
votes
0
answers
13
views
Data processing - how to input a pandas column that contains numbers and numpy arrays
I have a pandas df that contains numbers and strings. I use word2vec to convert all the strings into embeddings. The problem now is that these embeddings are all numpy arrays. So now my pandas df ...
- 1
1
vote
1
answer
108
views
Deleting part of saved XGBoost-Model (JSON) and reloading it
I am training an XGBoost-model on part of the ForestCover-Dataset. Then I save the trained model to json. Now I load the model and "update" the saved model with the data, that I previously ...
- 21
0
votes
1
answer
30
views
Validating ML regression model and predictions
I have a years worth of electricity power data on 15 minute intervals joined with weather data and time-of-week one hot dummy variables.
Is using train/test split an okay approach for validating the ...
- 403
1
vote
1
answer
25
views
0
votes
0
answers
27
views
How to optimize hyperparameters of Xgboost algorithm in multiple classes in R?
My unbalanced dataset with 4 classes. there is. I want to use xgboost algorithm for classification. But despite my research I couldn't find how to optimize hyperparameters for multiclass datasets. ...
- 1
1
vote
0
answers
275
views
Time series Classification - is LSTM better than XGBoost?
My question is whether LSTM RNN is a better predictor of a label (note not forecaster) than XGBoost. Thus far I have had moderate success with XGB but I wonder if the tree nature, random start and ...
- 111
0
votes
0
answers
64
views
Objective function in optuna seems to return at random points
I am trying to use a K-fold cross-validation within the optuna objective function.
Unfortunately, the output of the objective function seems to pop out at random places, not at the end of the cross-...
0
votes
1
answer
25
views
What do I make of all classification scores being equal to 1?
I've built an XGBoost classifier on a dataset that has 51 columns and a 1000 rows with following code:
...
- 103
0
votes
0
answers
123
views
Do we need to check the training score when we use randomizedsearchcv
given a model and a set of parameters, randomizedsearchCV(or gridsearchCV) gives the mean of the best scores from a list of different folds of the datasets.
Does the model control for overfitting? I ...
- 31
0
votes
1
answer
60
views
how do I test if overfitting exists when I use cross_val_score method?
I got the following code form a book on xgboost.
I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes.
mean accuracy is 81 which can be okay. but what if ...
- 31
1
vote
0
answers
53
views
XGBoost learning rate "rescaling according to size of trees". Explanation
I'm trying to understand the impact of the learning rate parameter in XGBoost. I started inspecting the source code. In this file, I found the following lines
...
0
votes
1
answer
501
views
6
votes
1
answer
183
views
How to choose between different models with similar results? RF, GLM and XGBoost
I am a medical doctor trying to make prediction models based on a database of approximately 1500 patients with 60+ parameters each.
I am dealing with a classification problem (mortality at 1, 3, 6 and ...
- 61
0
votes
0
answers
8
views
How to give a column a special weight for any algorithm? (binary option)
I'm dealing with binary option, I'm making a classification model using xgboost, my idea is to predict the posterior candle color by using some data of the 20 previous candles so I made a dataset on ...
