Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
650
questions
0
votes
0
answers
19
views
How can I write a custom loss function to punish lower predicted values?
I am trying to write a custom loss function for XGBRegressor that needs to punish predicted values that are under some arbitrary threshold. The code I came up with does not affect the results at all, ...
- 1
0
votes
1
answer
26
views
XGBoost training on sample of time series data
I am new to XGBoost and would like to use it on a time series dataset. Here is the scenario I'm faced with:
The data set contains N samples of length T, with N>>T. I'd like to train an XGBoost ...
- 101
3
votes
0
answers
46
views
Does ROC AUC different between crossval and test set indicate overfitting or other problem?
I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
- 106
1
vote
1
answer
51
views
Online Learning/Continual Learning for tree-based Algorithms
Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
- 113
0
votes
0
answers
7
views
Part random sample, part force sample with multiple event=0
Question: Is this approach sound?
Goal: Model probability of event = 1
Problem: Time series with multiple event = 0 per id, only one event=1
Approach:
Get all records where event = 1 (not random ...
- 1
0
votes
1
answer
12
views
combine xgb feature importance
Is it possible to combine the results from the xgb.boost importance function. For example, due to one hot encoding, I have a feature age=35 and another age=60. Is there a way that I can add these to ...
1
vote
1
answer
24
views
scale_pos_weight effect in XGBClassifier
I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier.
It says everywhere to set it to Count of negatives / Count of ...
- 161
0
votes
0
answers
25
views
XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression
i'm tring tune XGBoost model for my dataset.
...
0
votes
0
answers
29
views
Gradient tree boosting regularization term parameters
In the XGBoost documentation, they initially define the regularized objective as
\begin{equation}
\begin{split}
L & = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i) + \sum\limits_{i=1}^K \Omega(f_k)\\...
- 103
0
votes
1
answer
14
views
Gradient tree boosting additive training
In the XGBoost documentation, they specify that the additive training is done given an objective $obj^{(t)}$ defined as
$obj^{(t)} = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i^{(t-1)}+f_t(x_i)) + \sum\...
- 103
0
votes
0
answers
29
views
Setting number of features in xgboost's DMatrix construction
The data argument in DMatrix's constructor (from xgboost library) supports giving path to a ...
- 11
0
votes
0
answers
12
views
Time series forecasting model predicts increasing number for target variable when the actual values are zeroes
I am working on time series forecasting model, and I am using light gbm. The project goal is to predict the number of sales across different levels (very similar to the M5 competition). For instance, ...
- 113
0
votes
2
answers
36
views
What is the implication of having features with less variation in a tree based model?
I'm training a tree-based model (e.g. xgb). I have some features with more than 90% values constant. Does it add value to the model since the variation in the data is minimal?.
What would be the ...
- 231
0
votes
0
answers
13
views
Debug and address potential sources of errors in an xgboost ML model
I am training an xgboost ML model on dataset of shape ~3500x27. I had previously trained a model on about 1500 sample for market ...
- 302
1
vote
1
answer
29
views
Do best hyperparameters remain constant when data size is scaled?
Basically what the title is. The problem I currently have is that my dataset consists of 2.8 billion rows, and I have it as a Pyspark data frame.
I want to use some library such as FLAML for finding ...
- 121
2
votes
1
answer
151
views
Why XGboost does not work on small dataset
Here I am using Xgboost for classification for a simple small dataset, when x = 0 then y = 1 elif x = 1 then y = 0. Then I use the xgb.XGBClassifier() but the resulting probability is just 0.5. I ...
2
votes
2
answers
143
views
Are feature importances of ensemble methods sensible interpretable?
As mentioned in the question, it is easy to interpret the meaning of features in algorithms like simple decision trees. But in the case of ensemble methods that are known to average/modify features, ...
- 23
1
vote
0
answers
29
views
Is there a typo in the XGBoost example in the paper?
I am reading the XGBoost paper and I found this example below in the paper to calculate the quality tree structure:
To calculate the quality tree structure, we use the following formula:
My question,...
- 113
1
vote
1
answer
20
views
How to capture regularity and seasonality in purchase data?
I want to train a model on transaction data to predict whether a customer will buy the product in the next 90 days.
I observed seasonality in the data, i.e. during certain months of
the year, sale ...
- 231
1
vote
0
answers
18
views
How to convert machine learning XGBoost "R" binary model into CORE ML on iOS Swift?
How can we convert our sleep stage classification "R" XGBoost binary model on Windows into CORE ML on iOS to run model on iPhone?
CORE ML doc says it inputs XGBoost -- need guide or tutorial....
- 111
1
vote
1
answer
22
views
How can I manually add a root node in XGBoost tree(s)?
I have a binary feature {0, 1} and only when its value is 1, I would like my XGBoost model to "evaluate" a set of ...
- 11
0
votes
0
answers
12
views
Derivation of path dependent feature attributions in Tree SHAP
I was reading through the TreeSHAP paper by Lundberg & Lee. They proposed that every path can be considered an individual model and considering additivity property of SHAP - we can sum up the ...
0
votes
0
answers
56
views
How to Incorporate Upward Trend into XGBoost Time Series Forecasting
I'm working with an XGBoost XGBRegressor model right now, attempting to utilize it to predict time-series forecasted data. My dataset is not publicly available, so I will use general terms to describe ...
- 1
1
vote
1
answer
88
views
Linear regression returning negative values for house price prediction
I am trying to do a prediction of real estate (prices are in millions).
The mean price for the dataset is 4 million.
I do not have any negative values in my dataset,...
1
vote
0
answers
29
views
Error from XGBoost missing data handling
I have a regression problem with a very large dataset >50 million rows, 81 features and 1 target, all positive float values unevenly distributed between 0 - 1 million. I've trained an XGBoost model ...
- 36
1
vote
1
answer
133
views
XGboost Classifier predicits different results for same samples depending on test dataset size
I train a simple xgboost classifier model with the following lines.
...
- 41
0
votes
1
answer
36
views
Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?
In order to achieve scalable and robust time series forecast models, I am currently experimenting with metalearner ensembles.
Note, that I am also using a global modeling approach, so all time series ...
- 145
0
votes
0
answers
16
views
Is this XGBoost model tending to overfit?
Here is the list of hyperparameters that I used:
...
0
votes
0
answers
25
views
What should be the target vairable in CTR maximization problem?
I have a dataset that contains some user-specific detials like gender, age-range, region etc. and also the behavioural data which contains the historical click-through-rate (last 3 months) for ...
- 155
0
votes
1
answer
14
views
GridSearch multiplying the number of trees in XGboost?
I'm having an issue: after running an XGboost in a HalvingGridSearchCV, I receive a certain number of estimators (50 for example), but the number of trees is constantly being multiplied by 3. I don't ...
- 11
0
votes
1
answer
110
views
How to make XGBOOST capture trend in time series forecasting?
I am trying to forecast some sales data with monthly values, I have been trying some classical models as well ML models like XGBOOST.
My data with a feature set looks like this with a length of 110 ...
0
votes
0
answers
50
views
XGBClassifier's predictions are not probabilities with objective='binary:logistic'
I am using a XGBoost's XGBClassifier, a binary 0-1 target, and I am trying to define a custom metric function. It supposedly receives an array of predictions and a DMatrix with the training set ...
- 127
0
votes
0
answers
83
views
Ignoring features in XGBoost by setting them as "missing"
I have some data n x m and I want to ignore certain features. One idea I had is to mark those features as "missing", since XGBoost can handle missing ...
- 181
0
votes
0
answers
203
views
Interpretation of SHAP summary plot in a multi class context
I'm performing multi-class classification and uses SHAP values to interpret the features. I have 3 classes. I have testet XGBoost and Multinomial Logistic Regression. When i'm using XGBoost I am able ...
- 31
3
votes
1
answer
198
views
Is multicollinarity a problem when interpreting SHAP values from an XGBoost model?
I'm using an XGBoost model for multi-class classification and is looking at feature importance by using SHAP values. I'm curious if multicollinarity is a problem for the interpretation of the SHAP ...
- 31
0
votes
1
answer
68
views
Distribution of predicted probability is heavily skewed
I'm a beginner here. I'm just trying to use a xgboost method for classification learning problem. My data is 70-30 unbalanced. But I ran into a problem about the distribution of predicted probability ...
0
votes
0
answers
21
views
Imbalanced classification
I've tried all kind of oversampling undersampling techniques and I've tried also weighted Xgboost ( the model I'm trying to improve)
I couldn't surpass a very Bad F1 score : 0.09
What should I do
0
votes
0
answers
36
views
Make fitted xgboost or linear regression model predicts values in thé future
I have a machine learning model that I fitted with xgboost and linear regression. My dataset has thirteen features and has price as the target.
Is there any way to ...
0
votes
0
answers
16
views
Can we use an independent t-test as a metric for feature importance?
I have a supervised binary classification problem. I tuned an xgboost model on the training set and achieved a reasonably high accuracy on the test set. Now I want to interpret the results of the ...
- 165
1
vote
0
answers
76
views
Does it make sense to use target encoding together with tree-based models?
I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model).
Someone suggested to use target-encoding (mean/median of the ...
- 11
1
vote
1
answer
21
views
Random search grid not displaying scoring metric
I want to do a grid search of some few hyperparameters through a XGBClassifier of a binary class, but whenever i run it the score value (roc_auc) is not being ...
- 111
0
votes
0
answers
58
views
Custom multi-label cross-entropy loss that boosts weight of particular errors
I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
- 1
0
votes
0
answers
82
views
Decision tree from boosted tree regressor Google bigquery ML
If I set the num_parallel tree to 1 and max_iteration to 1 in boosted_tree_regressor of Google Big Query ML will it work as Decision tree regressor ?
Also can such decision tree give negative ...
- 379
0
votes
1
answer
61
views
ROC-AUC Imbalanced Data Score Interpretation
I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations.
The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
- 21
0
votes
1
answer
55
views
Meassage "AUC-PR: the dataset only contains pos or neg samples"
My goal is to fit column name 'My_Val' using columns 'B1','B2', I tried XGboost function as below. And "AUC-PR: the dataset only contains pos or neg samples" shows. I have no idea what went ...
- 105
0
votes
1
answer
26
views
How to provide Intentional Bias towards recent examples in Text Classification?
I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
1
vote
1
answer
34
views
XGBoost results changing when one row is removed
I have a training dataset of 2,600 rows and 26 columns.
I trained an XGBoost (1.3.1) Classification model using the data and evaluated it using a test set of c. 800 rows.
Whilst experimenting I found ...
- 21
0
votes
1
answer
77
views
XGBoost regression scale invariant? 0 depth trees for target variable with small (1E-7) values
I thought the consensus was that XGBoost was largely scale-invariant and scaling of features isn't really necessary but something's going wrong and I don't understand what.
I have a range of features ...
- 103
2
votes
1
answer
63
views
Learning to Rank with Unlabelled Dataset
I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
- 51
0
votes
0
answers
20
views
Searching machine learning algorithm for regression problem with many features
I have a machine learning problem with about 160 features and 400 cases and I want to find the best predictors for a continuous outcome. The dataset contains variables of psychotherapists and clients. ...
