Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Predict best score on unlabelled test set

Data I have one dataset with $1500$ data points, each with $\sim 23 000$ features (gene expression data, if that matters). However, I've split this dataset into a labelled training set of size 1000, ...
1
vote
0answers
26 views

Identifying possible data leakage

I am building a binary classification model for imbalanced dataset using XGBoost. I tuned the hyperparameters for four different models based on 2 training datasets and 2 optimization metrics. Class ...
0
votes
0answers
10 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
0
votes
0answers
66 views

ExtraTreeClassifier does not handle missing values

I am using sklearn.tree.ExtraTreeClassifier. It does not handle missing value in training data. All tree-based algorithms handle missing value internally. So, is ...
2
votes
2answers
108 views

XGBoost and Random Forest: ntrees vs. number of boosting rounds vs. n_estimators

So I understand the main difference between Random Forests and GB Methods. Random Forests grow parallel trees and GB Methods grow one tree for each iteration. However, I am confused on the vocab used ...
0
votes
0answers
22 views

L1 vs. L2 Loss in XGBoost [duplicate]

I understand the concepts of L1 and L2 loss (i.e. L1 loss will force some parameter coefficients to zero while L2 will only make them approach zero). What do these do when implemented in XGBoost? Does ...
1
vote
0answers
16 views

Multi-classification: low precision due to imbalanced classes in test data - what to do?

I built a multi-classification model with 3 result classes (XGBoost using R's caret-package): A, B and C. I undersampled my training data - so every class is equally abundant for training. The ...
1
vote
0answers
29 views

Every machine learing model i build, always predict wrongly almost the same samples. (Random forest, XGBoost, AdaBoost)

First of all, I'd like to apologize for any spelling or grammar mistakes. I'm having a problem using R for a classification problem. My dataset contains ~300.000 genomic data, and the features are ...
2
votes
1answer
40 views

High error machine learning regressor algorithm in Python - XGBOOST Regressor

I have a dataframe with real state data from florida, it includes single apartments and buildings data: ...
1
vote
2answers
30 views

XGBoost Objective is Changed

I am trying to use XGBoost in python for logistic regression. I am calling it as follows ...
2
votes
1answer
45 views

treeExplainer algorithm intuition

I'm reading the paper about the treeExplainer; the pseudo-code of Algorithm 1 is a bit cryptical as most of the variables are not even defined (same with sampling and all details involved). Is there a ...
5
votes
2answers
122 views

Optimising for Brier objective function directly gives worse Brier score than optimising with custom objective - what does it tell me?

I am training an XGBoost model and as I care the most about resulting probabilities, not classification itself I have chosen Brier score as a metric for my model, so that probabilities would be well ...
5
votes
2answers
139 views

Understanding XG Boost Training (Multi class classification)

I have been working with XG boost for classification (multi class classification : 6 classes) I use 5 fold CV to train and validate my model. Please refer to the ...
1
vote
0answers
26 views

Using a Subset of Categories in a Categorical Column

I have a XGBoost model and I'm going to retrain it by adding new features. There is a column in my data and it's about professions of the customers. It has 60 categories. I suppose there is no need to ...
0
votes
1answer
131 views

How to use r2-score as a loss function in LightGBM?

I am trying to implement a custom loss function in LightGBM for a regression problem. The intrinsic metrics do not help me much, because they penalise for outliers... Is there any way to use ...
0
votes
0answers
72 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
0
votes
1answer
38 views

Can you combine two xgboost models into one?

If you have built two different xgbost models, with say 100 trees each, is it possible to combine into an xgboost model with 200 trees?
1
vote
0answers
54 views

State of the Art/Research 2020 of Time Series Forecasting/Prediction

Im looking for the state of the art/research of time series data for forcasting/prediction. As far as im aware it is Extrem Gradient Boosting (XGBoost) or LSTM (neuronal networks) or are there other ...
1
vote
0answers
64 views

Gradient boosting Regression with zero-inflated outcome

I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
0
votes
1answer
73 views

What is the selection criteria to choose between XGBoost and Random Forest

I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when ...
1
vote
1answer
86 views

XGBoost multiclassification interpreting predicted probabilities

Let's consider an example. I have patient level data, their symptoms, reading from various medical tests. Based on that, I have built a binary classifier given patient data to classify if they are ...
0
votes
0answers
44 views

XGBoost Python API - performance by API, R vs Python inconsistencies, GPU memory and verbosity

I'm combining a few questions together as I feel that it could benefit others. XGBoost API Is there a performance boost (training time or accuracy) when you use the learning API vs the Scikit-Learn ...
7
votes
1answer
195 views

What is query id (“qid”) in XGBoost

In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet: ...
1
vote
2answers
64 views

Weights for unbalanced classification

I'm working with an unbalanced classification problem, in which the target variable contains: np.bincount(y_train) array([151953, 13273]) i.e. ...
1
vote
0answers
21 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
1
vote
1answer
225 views

xgboost classifier predicted negative probabilities

I'm using XGBoost for a binary classification problem. There is no negative label, only 1 and 0. I tunned the hyperparameters using Bayesian optimization then tried to train the final model with the ...
1
vote
2answers
30 views

scale_pos_weight using XGBoost's Learning API

I see it is possible to add a weight for unbalanced problems in XGBoost's Scikit-Learn API through scale_pos_weight. Does it have an equivalent in the Learning API? If not, is there a reason behind ...
5
votes
2answers
232 views

XGBOOST - different result between train_test_split and manually splitting

I am trying to train XGBOOST model. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y) when I'm using ...
2
votes
1answer
89 views

Interpretable xgboost - Calculate cover feature importance

When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance. There are same parameters in the xgb api such as: weight, gain, cover, ...
3
votes
1answer
129 views

Small number of estimators in gradient boosting

I am tuning a regression gradient boosting-based model to determine the appropriate hyperparameters using 4-folds cross validation. More specifically, I am using XGBoost and lightGBM for the models ...
-1
votes
1answer
21 views

Machine Learning Out of test data forecast (XGBoost, ANN)

I see a lot of applications for machine learning techniques applied to time series. Unfortunately almost all kernels with XGBoost or ANN stop short in creating an actual forecast. The achieve a great ...
0
votes
0answers
27 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
2
votes
1answer
58 views

Each time the result of XGBoost is different

I'm running xgboost and I fix the seed number but the result is different for each time when I rerun the xgboost with same dataset ...
1
vote
0answers
21 views

xgboost result is not stable [closed]

Here is my xgboost hyperparameters ...
1
vote
0answers
25 views

Multi-label classification yielding too much unlabeled rows

I am performing multi-label classification with xgboost + OneVsRestClassifier from sklearn. ...
1
vote
1answer
35 views

feature importance and xgboost?

Let say I got feature importance for xgclassifier ...
1
vote
0answers
43 views

is there metric 'multi_logloss' for xgb crassifier?

lgb has the log_loss metric ...
1
vote
2answers
22 views

is there cross validation for xgb classification for multi labels?

is there cross validation for xgb classification for multi labels? I have been search but can not find any cross validation for xgb classifier is using cross validation for xgb or xgb classifier ...
1
vote
1answer
18 views

is there any rule to apply pca to the imbalance data? [closed]

Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost) I used multiclass imbalance data to pca but the log-loss accuracy getting decrease any theoritical background of this?
0
votes
0answers
13 views

Getting bad predictions for high true values of target variable

I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
1answer
296 views

XGBoost: How to set the probability threshold for multi class classification

I am using the XGBoost for classification of text data. There are the 3 different classes in the training dataset. ...
0
votes
1answer
37 views

order of features importance after make_column_transformer and pipeline

I have a data preparation and model fitting pipeline that takes a dataframe (X_trn) and uses the ‘make_column_transformer’ and ‘Pipeline’ functions in sklearn to prepare the data and fit XGBRegressor. ...
1
vote
1answer
35 views

order of features for model tuning vs model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
4
votes
1answer
193 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
5
votes
1answer
176 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
1
vote
2answers
34 views

Can an entire data frame be used as a prediction variable?

I am attempting to use XGBoost in R to train a model that predicts a fixed number of target variables using all data from previous dates, as well as the two categorical variables (...
1
vote
1answer
34 views

How to check for “statistical significance” of categorical feature in black box models

Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
2
votes
1answer
103 views

How to calculate joint feature contribution for XGBoost Classifier in python?

I referred to http://savvastjortjoglou.com/intrepretable-machine-learning-nfl-combine.html#Joint-Feature-Contributions this beautiful document to research about joint feature contibutions. But this ...
4
votes
1answer
118 views

XGBoost: how to adjust the probabilities of a binary classifier to match training data?

Training and testing data have around 1% positives, but the model predicts only around 0.1% as positives. The model is an xgboost classifier. I’ve tried calibration but it didn’t improve much. I ...

1
2
3 4 5
10