Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

How to deal with missing values in the output for XGBoost Regressor

So I am trying to create a regression model that takes two arrays as the input features and an array as an Output. However, some of the point in this dataset do not contain any value. This is because ...
RM25's user avatar
  • 1
0 votes
0 answers
23 views

Custom objective function for xgboost to optimize lift in best decile

I've tried to define a custom objective function for xgboost to optimize the lift for a binary classification problem in the upper decile. The task is simply to concentrate the training effort to the ...
Arne Bøckmann's user avatar
0 votes
1 answer
102 views

Understand and compute confidence interval and coefficient of variation for regression model

I would like to better understand the concepts of: coefficient of variation and confidence interval. Trivially taking the definitions from wikipedia: confidence interval (CI) In frequentist ...
Cata's user avatar
  • 1
0 votes
0 answers
23 views

Ordinal log-loss in a multiclass classification in XGBoost?

I have a multi-class problem that which classes are simultaneously mutually exclusive and have ordering. You can think of the classes as being some score: 0 (Low), 1 (Medium), 2 (High). What I would ...
deblue's user avatar
  • 111
0 votes
0 answers
36 views

Making ML Model (XGBoost) Output Smoother

I am using XGBoost for a classification problem. The model has multiple inputs and a probability as output. For simplicity's sake, let's say the model only has one continous input and the function to ...
Neo's user avatar
  • 11
0 votes
0 answers
5 views

Histogram creation in lightgbm in the train API and the scikit-learn API. Is it always benefitial to use the train API?

In the LightGBM for python we have a scikit-learn API in which (either for regression or for classification) there is fit method whose documentation is fit(X, y, sample_weight=None, init_score=None, ...
figs_and_nuts's user avatar
0 votes
0 answers
14 views

Xgboost giving different results on windows and Linux

I'm facing one issue with the results not matching on Windows and Linux. I'm using the same code on both machines. Using separate trains. csv and test. csv The problem is due to sampling and ...
Sagar J's user avatar
0 votes
1 answer
24 views

Validity of using raw time series data for training of xgboost/random forest classifier

I am currently working on a project aiming of classification of process states based on time series data. For this, we are looking at different models, such as XGBoost-based classifiers or ...
arc_lupus's user avatar
  • 103
0 votes
0 answers
15 views

Minimize MAE loss for a target that is sum of two other targets

Working on a regression modelling task where my dataset have some feature columns, two more columns A, B and a target column T. The goal is to predict T, and minimize MAE, that is ...
Ngo Cuong's user avatar
0 votes
0 answers
21 views

Binary classification using xgboost

Why when adding new features in my ADS for a binary classification using XGBOOST my score and uplift has decreased ? What is the best way to treate categorical features or other features in order that ...
Warda_IDRIS's user avatar
0 votes
0 answers
4 views

Parameter outside specified range using HGboost/Hyperopt library

I am trying to use the HGboost library which uses the Hyperopt library for doing hyperparameter optimization of an XGboost model. The script runs fine but the optimized parameter for "...
AWGIS's user avatar
  • 101
1 vote
2 answers
82 views

Xgboost model predicting extreme values for events and non-events | Overfitting

Extreme values are predicted by my trained xgboost classification model in BQML for both events (Y=1) and non-events (Y=0). For all event observations, the model calculates probability scores that ...
Scott Grammilo's user avatar
0 votes
0 answers
20 views

XGBoost Classifier Evaluation Confusion on New Dataset Despite High Cross-Validation Scores

I have built an XGBoost classifier model with 90 features, trained on a dataset containing 760k samples. I took great care to separate the labels from the features in both the training and testing ...
oklen's user avatar
  • 1
1 vote
0 answers
33 views

Understanding lgbm histogram building

...
figs_and_nuts's user avatar
0 votes
0 answers
31 views

XGBoost Classifier + Isotonic Regression leading to worse probability accuracy

I'm testing out an XGBoost Classifier with the goal of using the probabilities it predicts in production. I know that tree based model probabilities are often not calibrated well so I decided to test ...
Ted's user avatar
  • 1
0 votes
0 answers
46 views

XGBoost tweedie regression objetive from scratch

I'm trying to gain a deeper understanding of the tweedie loss function and how it is used in XGBoost. So, I tried to implement it from scratch. I started by examining the original implementation. I ...
Diadochokinetic's user avatar
0 votes
0 answers
40 views

XGBRegressor underestimates sum of regression target (insurance application)

I'm trying to learn how to apply Boostig Algorithms (e.g. XGBoost) to insurance applications (premium calculation). As a starting point I used this tutorial from the scikit-learn website. tldr: The ...
Diadochokinetic's user avatar
1 vote
1 answer
50 views

Quality of first tree prediction in xgboost

Up until now I have mostly been using neural nets for regression. There on each iteration (epoch) the weights are updated using the whole training data, and we expect the error to go down each time, ...
SBF's user avatar
  • 111
0 votes
0 answers
44 views

Problem calculating f-1 score when when running hyperparameter tuning using Optuna

I'm trying to run Stratified CV hyperparameter tuning on an XGBClassifier using Optuna. This is a multiclass problem with 3 classes (labelled 0 through 2 in the "classes" variable). These ...
Metrician's user avatar
1 vote
0 answers
43 views

Replication of XGBoost's binary:logistic loss

I am trying to replicate XGBoost's logistic loss function as a first step before implementing my own custom loss functions. Following from here and looking at the original code in git repository, I ...
deblue's user avatar
  • 111
1 vote
1 answer
51 views

Time series prediction on test set produces multiple cycles instead of following trend

I have a non-stationary time series where I am trying to build a model for forecasting. So far on test set it produces multiple cycles no matter which technique I use. There's just one feature ...
Thermal_insulator's user avatar
0 votes
1 answer
60 views

XGBoost with objective as binary:logistic returning only binary value

I am training an xgboost model for binary classification using objective as 'binary:logistic'. The model should predict probability but it is outputting either 0 or 1. I want the model to output the ...
shivani's user avatar
  • 120
1 vote
3 answers
85 views

Feature importance score for a feature that contains mostly 0's in XGBoost

I have read that the feature importance scores are calculated based on how a split on that feature improves performance. I have a binary classification dataset and am running XGBoost classifier on it. ...
Vjs's user avatar
  • 11
0 votes
1 answer
45 views

eval_metric of XGBoost // ML model in general

Say I am using Xgboost on a binary classification task. eval_metric is one of the model parameter. How should I think about the impact of using different eval_metric(e.g rmse/mae/logloss) in general? ...
pathtoagi's user avatar
0 votes
0 answers
53 views

Can I decompose SHAP interaction values like a linear regression?

I had a question regarding the shap interaction matrix. Suppose I have 500 samples with 2 features. Then my interaction matrix becomes (500,2,2). I want to calculate the SHAP values of each feature ...
cwanderroycbooks's user avatar
0 votes
0 answers
47 views

Build a model with cross-validation on entire dataset to learn insights?

Goal : Use XGBoost regression to learn insights from data. Prediction or forecasting not needed. Hypothesis : If the model fits the entire dataset well, it can maybe capture its "physics" in ...
cwanderroycbooks's user avatar
0 votes
0 answers
54 views

XGBoost Architecture Diagram required

Good Day! My topic is general and theory related, about XGBoost working. I am searching XGBoost Architecture Diagram. I know it works on principles of Decision Trees, Bagging, Random Forest, Boosting, ...
P_Z's user avatar
  • 1
0 votes
0 answers
31 views

weight derivation for XGBOOST in regression trees

I am working on a variation of the XGBOOST presented on the following paper by Chen and Guestrin: https://dl.acm.org/doi/abs/10.1145/2939672.2939785 My goal is to fit a linear regression in each leaf ...
firstname's user avatar
0 votes
0 answers
41 views

How to predict an xgboost model outcome directly from the trained trees?

I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it ...
Subhajit Saha's user avatar
1 vote
1 answer
118 views

Funny looking learning curve

I am training an XGBoost for binary classification. I am getting a very odd-looking learning curve. Any idea what is going on and how I might fix this? Thanks
pyassign67's user avatar
1 vote
1 answer
131 views

Why is accuracy score suddenly becoming 1 on using XGBoost?

I am developing a music classification system based on a kaggle dataset: https://www.kaggle.com/datasets/vatsalmavani/spotify-dataset I tried using K means classifier to classify the songs into 4 ...
fat_gladiator17's user avatar
0 votes
0 answers
36 views

Why am I getting differing "gain" feature importances from XGBoost?

I am currently training an XGBoost model for binary classification. I have fitted and predicted with the model but when I try to get the "gain" type feature importances, the results differ ...
user19941466's user avatar
0 votes
0 answers
116 views

How do the predict() and predict_proba() for xgbClassifier() work?

I am using xgboost classifier for a binary classification problem. I used the predict() to get the class predictions (0/1) but I also used the predict_proba() method to get the class probabilities (my ...
user62198's user avatar
  • 1,091
0 votes
1 answer
221 views

How to intrepret low F1 score and high AUC on training set?

I am currently working on a very imbalanced dataset: 24 million transactions (rows of data) 30,000 fraudulent transactions (0.1% of total transactions) and I am using XGBoost as the model to predict ...
Hai Nguyen's user avatar
0 votes
0 answers
47 views

Why does my accuracy score drop after hyperparameter tuning in XGBoost?

I am trying to tune the model I've built, but every time I change hyperparameters my accuracy score drops significantly. I'm using RandomizedSearchCV and best_params_ to determine which parameters I ...
mvinegret's user avatar
0 votes
0 answers
76 views

Visualize Catboost and XGBoost training process + Cross Validation

I want to optimize Catboost and XGBoost models and visualize this process such that: Use 3-fold cross-validation Use my own pre-processing pipeline (Missing value imputation, over- or undersampling) ...
Ars ML's user avatar
  • 61
0 votes
0 answers
13 views

Data processing - how to input a pandas column that contains numbers and numpy arrays

I have a pandas df that contains numbers and strings. I use word2vec to convert all the strings into embeddings. The problem now is that these embeddings are all numpy arrays. So now my pandas df ...
cyc's user avatar
  • 1
1 vote
1 answer
108 views

Deleting part of saved XGBoost-Model (JSON) and reloading it

I am training an XGBoost-model on part of the ForestCover-Dataset. Then I save the trained model to json. Now I load the model and "update" the saved model with the data, that I previously ...
dalefi's user avatar
  • 21
0 votes
1 answer
30 views

Validating ML regression model and predictions

I have a years worth of electricity power data on 15 minute intervals joined with weather data and time-of-week one hot dummy variables. Is using train/test split an okay approach for validating the ...
bbartling's user avatar
  • 403
1 vote
1 answer
25 views

XGBoost prints trees beyond n_estimator

I have a XGBoost model with the following parameters ...
Itminan's user avatar
  • 11
0 votes
0 answers
27 views

How to optimize hyperparameters of Xgboost algorithm in multiple classes in R?

My unbalanced dataset with 4 classes. there is. I want to use xgboost algorithm for classification. But despite my research I couldn't find how to optimize hyperparameters for multiclass datasets. ...
ali's user avatar
  • 1
1 vote
0 answers
275 views

Time series Classification - is LSTM better than XGBoost?

My question is whether LSTM RNN is a better predictor of a label (note not forecaster) than XGBoost. Thus far I have had moderate success with XGB but I wonder if the tree nature, random start and ...
James Oliver's user avatar
0 votes
0 answers
64 views

Objective function in optuna seems to return at random points

I am trying to use a K-fold cross-validation within the optuna objective function. Unfortunately, the output of the objective function seems to pop out at random places, not at the end of the cross-...
Adrian Mureșan's user avatar
0 votes
1 answer
25 views

What do I make of all classification scores being equal to 1?

I've built an XGBoost classifier on a dataset that has 51 columns and a 1000 rows with following code: ...
Somanna's user avatar
  • 103
0 votes
0 answers
123 views

Do we need to check the training score when we use randomizedsearchcv

given a model and a set of parameters, randomizedsearchCV(or gridsearchCV) gives the mean of the best scores from a list of different folds of the datasets. Does the model control for overfitting? I ...
Mehmet Deniz's user avatar
0 votes
1 answer
60 views

how do I test if overfitting exists when I use cross_val_score method?

I got the following code form a book on xgboost. I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes. mean accuracy is 81 which can be okay. but what if ...
Mehmet Deniz's user avatar
1 vote
0 answers
53 views

XGBoost learning rate "rescaling according to size of trees". Explanation

I'm trying to understand the impact of the learning rate parameter in XGBoost. I started inspecting the source code. In this file, I found the following lines ...
Pablo Brianese's user avatar
0 votes
1 answer
501 views

XGB DataFrame.dtypes error

Here is the code to assign the variables ...
Nata's user avatar
  • 3
6 votes
1 answer
183 views

How to choose between different models with similar results? RF, GLM and XGBoost

I am a medical doctor trying to make prediction models based on a database of approximately 1500 patients with 60+ parameters each. I am dealing with a classification problem (mortality at 1, 3, 6 and ...
user145725's user avatar
0 votes
0 answers
8 views

How to give a column a special weight for any algorithm? (binary option)

I'm dealing with binary option, I'm making a classification model using xgboost, my idea is to predict the posterior candle color by using some data of the 20 previous candles so I made a dataset on ...
Davi Américo's user avatar

1
2 3 4 5
14