Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

How can I write a custom loss function to punish lower predicted values?

I am trying to write a custom loss function for XGBRegressor that needs to punish predicted values that are under some arbitrary threshold. The code I came up with does not affect the results at all, ...
0 votes
1 answer
26 views

XGBoost training on sample of time series data

I am new to XGBoost and would like to use it on a time series dataset. Here is the scenario I'm faced with: The data set contains N samples of length T, with N>>T. I'd like to train an XGBoost ...
3 votes
0 answers
46 views

Does ROC AUC different between crossval and test set indicate overfitting or other problem?

I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
1 vote
1 answer
51 views

Online Learning/Continual Learning for tree-based Algorithms

Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
0 votes
0 answers
7 views

Part random sample, part force sample with multiple event=0

Question: Is this approach sound? Goal: Model probability of event = 1 Problem: Time series with multiple event = 0 per id, only one event=1 Approach: Get all records where event = 1 (not random ...
  • 1
0 votes
1 answer
12 views

combine xgb feature importance

Is it possible to combine the results from the xgb.boost importance function. For example, due to one hot encoding, I have a feature age=35 and another age=60. Is there a way that I can add these to ...
1 vote
1 answer
24 views

scale_pos_weight effect in XGBClassifier

I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier. It says everywhere to set it to Count of negatives / Count of ...
  • 161
0 votes
0 answers
25 views

XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression

i'm tring tune XGBoost model for my dataset. ...
0 votes
0 answers
29 views

Gradient tree boosting regularization term parameters

In the XGBoost documentation, they initially define the regularized objective as \begin{equation} \begin{split} L & = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i) + \sum\limits_{i=1}^K \Omega(f_k)\\...
  • 103
0 votes
1 answer
14 views

Gradient tree boosting additive training

In the XGBoost documentation, they specify that the additive training is done given an objective $obj^{(t)}$ defined as $obj^{(t)} = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i^{(t-1)}+f_t(x_i)) + \sum\...
  • 103
0 votes
0 answers
29 views

Setting number of features in xgboost's DMatrix construction

The data argument in DMatrix's constructor (from xgboost library) supports giving path to a ...
  • 11
0 votes
0 answers
12 views

Time series forecasting model predicts increasing number for target variable when the actual values are zeroes

I am working on time series forecasting model, and I am using light gbm. The project goal is to predict the number of sales across different levels (very similar to the M5 competition). For instance, ...
  • 113
0 votes
2 answers
36 views

What is the implication of having features with less variation in a tree based model?

I'm training a tree-based model (e.g. xgb). I have some features with more than 90% values constant. Does it add value to the model since the variation in the data is minimal?. What would be the ...
  • 231
0 votes
0 answers
13 views

Debug and address potential sources of errors in an xgboost ML model

I am training an xgboost ML model on dataset of shape ~3500x27. I had previously trained a model on about 1500 sample for market ...
  • 302
1 vote
1 answer
29 views

Do best hyperparameters remain constant when data size is scaled?

Basically what the title is. The problem I currently have is that my dataset consists of 2.8 billion rows, and I have it as a Pyspark data frame. I want to use some library such as FLAML for finding ...
2 votes
1 answer
151 views

Why XGboost does not work on small dataset

Here I am using Xgboost for classification for a simple small dataset, when x = 0 then y = 1 elif x = 1 then y = 0. Then I use the xgb.XGBClassifier() but the resulting probability is just 0.5. I ...
2 votes
2 answers
143 views

Are feature importances of ensemble methods sensible interpretable?

As mentioned in the question, it is easy to interpret the meaning of features in algorithms like simple decision trees. But in the case of ensemble methods that are known to average/modify features, ...
  • 23
1 vote
0 answers
29 views

Is there a typo in the XGBoost example in the paper?

I am reading the XGBoost paper and I found this example below in the paper to calculate the quality tree structure: To calculate the quality tree structure, we use the following formula: My question,...
  • 113
1 vote
1 answer
20 views

How to capture regularity and seasonality in purchase data?

I want to train a model on transaction data to predict whether a customer will buy the product in the next 90 days. I observed seasonality in the data, i.e. during certain months of the year, sale ...
  • 231
1 vote
0 answers
18 views

How to convert machine learning XGBoost "R" binary model into CORE ML on iOS Swift?

How can we convert our sleep stage classification "R" XGBoost binary model on Windows into CORE ML on iOS to run model on iPhone? CORE ML doc says it inputs XGBoost -- need guide or tutorial....
  • 111
1 vote
1 answer
22 views

How can I manually add a root node in XGBoost tree(s)?

I have a binary feature {0, 1} and only when its value is 1, I would like my XGBoost model to "evaluate" a set of ...
  • 11
0 votes
0 answers
12 views

Derivation of path dependent feature attributions in Tree SHAP

I was reading through the TreeSHAP paper by Lundberg & Lee. They proposed that every path can be considered an individual model and considering additivity property of SHAP - we can sum up the ...
0 votes
0 answers
56 views

How to Incorporate Upward Trend into XGBoost Time Series Forecasting

I'm working with an XGBoost XGBRegressor model right now, attempting to utilize it to predict time-series forecasted data. My dataset is not publicly available, so I will use general terms to describe ...
1 vote
1 answer
88 views

Linear regression returning negative values for house price prediction

I am trying to do a prediction of real estate (prices are in millions). The mean price for the dataset is 4 million. I do not have any negative values in my dataset,...
1 vote
0 answers
29 views

Error from XGBoost missing data handling

I have a regression problem with a very large dataset >50 million rows, 81 features and 1 target, all positive float values unevenly distributed between 0 - 1 million. I've trained an XGBoost model ...
  • 36
1 vote
1 answer
133 views

XGboost Classifier predicits different results for same samples depending on test dataset size

I train a simple xgboost classifier model with the following lines. ...
  • 41
0 votes
1 answer
36 views

Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?

In order to achieve scalable and robust time series forecast models, I am currently experimenting with metalearner ensembles. Note, that I am also using a global modeling approach, so all time series ...
0 votes
0 answers
16 views

Is this XGBoost model tending to overfit?

Here is the list of hyperparameters that I used: ...
0 votes
0 answers
25 views

What should be the target vairable in CTR maximization problem?

I have a dataset that contains some user-specific detials like gender, age-range, region etc. and also the behavioural data which contains the historical click-through-rate (last 3 months) for ...
  • 155
0 votes
1 answer
14 views

GridSearch multiplying the number of trees in XGboost?

I'm having an issue: after running an XGboost in a HalvingGridSearchCV, I receive a certain number of estimators (50 for example), but the number of trees is constantly being multiplied by 3. I don't ...
0 votes
1 answer
110 views

How to make XGBOOST capture trend in time series forecasting?

I am trying to forecast some sales data with monthly values, I have been trying some classical models as well ML models like XGBOOST. My data with a feature set looks like this with a length of 110 ...
0 votes
0 answers
50 views

XGBClassifier's predictions are not probabilities with objective='binary:logistic'

I am using a XGBoost's XGBClassifier, a binary 0-1 target, and I am trying to define a custom metric function. It supposedly receives an array of predictions and a DMatrix with the training set ...
0 votes
0 answers
83 views

Ignoring features in XGBoost by setting them as "missing"

I have some data n x m and I want to ignore certain features. One idea I had is to mark those features as "missing", since XGBoost can handle missing ...
0 votes
0 answers
203 views

Interpretation of SHAP summary plot in a multi class context

I'm performing multi-class classification and uses SHAP values to interpret the features. I have 3 classes. I have testet XGBoost and Multinomial Logistic Regression. When i'm using XGBoost I am able ...
3 votes
1 answer
198 views

Is multicollinarity a problem when interpreting SHAP values from an XGBoost model?

I'm using an XGBoost model for multi-class classification and is looking at feature importance by using SHAP values. I'm curious if multicollinarity is a problem for the interpretation of the SHAP ...
0 votes
1 answer
68 views

Distribution of predicted probability is heavily skewed

I'm a beginner here. I'm just trying to use a xgboost method for classification learning problem. My data is 70-30 unbalanced. But I ran into a problem about the distribution of predicted probability ...
0 votes
0 answers
21 views

Imbalanced classification

I've tried all kind of oversampling undersampling techniques and I've tried also weighted Xgboost ( the model I'm trying to improve) I couldn't surpass a very Bad F1 score : 0.09 What should I do
0 votes
0 answers
36 views

Make fitted xgboost or linear regression model predicts values in thé future

I have a machine learning model that I fitted with xgboost and linear regression. My dataset has thirteen features and has price as the target. Is there any way to ...
0 votes
0 answers
16 views

Can we use an independent t-test as a metric for feature importance?

I have a supervised binary classification problem. I tuned an xgboost model on the training set and achieved a reasonably high accuracy on the test set. Now I want to interpret the results of the ...
  • 165
1 vote
0 answers
76 views

Does it make sense to use target encoding together with tree-based models?

I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model). Someone suggested to use target-encoding (mean/median of the ...
  • 11
1 vote
1 answer
21 views

Random search grid not displaying scoring metric

I want to do a grid search of some few hyperparameters through a XGBClassifier of a binary class, but whenever i run it the score value (roc_auc) is not being ...
0 votes
0 answers
58 views

Custom multi-label cross-entropy loss that boosts weight of particular errors

I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
0 votes
0 answers
82 views

Decision tree from boosted tree regressor Google bigquery ML

If I set the num_parallel tree to 1 and max_iteration to 1 in boosted_tree_regressor of Google Big Query ML will it work as Decision tree regressor ? Also can such decision tree give negative ...
0 votes
1 answer
61 views

ROC-AUC Imbalanced Data Score Interpretation

I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations. The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
0 votes
1 answer
55 views

Meassage "AUC-PR: the dataset only contains pos or neg samples"

My goal is to fit column name 'My_Val' using columns 'B1','B2', I tried XGboost function as below. And "AUC-PR: the dataset only contains pos or neg samples" shows. I have no idea what went ...
  • 105
0 votes
1 answer
26 views

How to provide Intentional Bias towards recent examples in Text Classification?

I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
1 vote
1 answer
34 views

XGBoost results changing when one row is removed

I have a training dataset of 2,600 rows and 26 columns. I trained an XGBoost (1.3.1) Classification model using the data and evaluated it using a test set of c. 800 rows. Whilst experimenting I found ...
0 votes
1 answer
77 views

XGBoost regression scale invariant? 0 depth trees for target variable with small (1E-7) values

I thought the consensus was that XGBoost was largely scale-invariant and scaling of features isn't really necessary but something's going wrong and I don't understand what. I have a range of features ...
  • 103
2 votes
1 answer
63 views

Learning to Rank with Unlabelled Dataset

I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
  • 51
0 votes
0 answers
20 views

Searching machine learning algorithm for regression problem with many features

I have a machine learning problem with about 160 features and 400 cases and I want to find the best predictors for a continuous outcome. The dataset contains variables of psychotherapists and clients. ...

1
2 3 4 5
13