Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
624
questions
0
votes
0
answers
12
views
Imbalanced classification
I've tried all kind of oversampling undersampling techniques and I've tried also weighted Xgboost ( the model I'm trying to improve)
I couldn't surpass a very Bad F1 score : 0.09
What should I do
0
votes
0
answers
14
views
Make fitted xgboost or linear regression model predicts values in thé future
I have a machine learning model that I fitted with xgboost and linear regression. My dataset has thirteen features and has price as the target.
Is there any way to ...
0
votes
0
answers
13
views
Can we use an independent t-test as a metric for feature importance?
I have a supervised binary classification problem. I tuned an xgboost model on the training set and achieved a reasonably high accuracy on the test set. Now I want to interpret the results of the ...
- 165
1
vote
0
answers
46
views
Does it make sense to use target encoding together with tree-based models?
I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model).
Someone suggested to use target-encoding (mean/median of the ...
- 11
1
vote
1
answer
15
views
Random search grid not displaying scoring metric
I want to do a grid search of some few hyperparameters through a XGBClassifier of a binary class, but whenever i run it the score value (roc_auc) is not being ...
- 111
0
votes
0
answers
7
views
Custom multi-label cross-entropy loss that boosts weight of particular errors
I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
- 1
0
votes
0
answers
19
views
Decision tree from boosted tree regressor Google bigquery ML
If I set the num_parallel tree to 1 and max_iteration to 1 in boosted_tree_regressor of Google Big Query ML will it work as Decision tree regressor ?
Also can such decision tree give negative ...
- 379
0
votes
1
answer
17
views
ROC-AUC Imbalanced Data Score Interpretation
I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations.
The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
- 21
0
votes
1
answer
12
views
Meassage "AUC-PR: the dataset only contains pos or neg samples"
My goal is to fit column name 'My_Val' using columns 'B1','B2', I tried XGboost function as below. And "AUC-PR: the dataset only contains pos or neg samples" shows. I have no idea what went ...
- 105
0
votes
1
answer
23
views
How to provide Intentional Bias towards recent examples in Text Classification?
I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
1
vote
1
answer
30
views
XGBoost results changing when one row is removed
I have a training dataset of 2,600 rows and 26 columns.
I trained an XGBoost (1.3.1) Classification model using the data and evaluated it using a test set of c. 800 rows.
Whilst experimenting I found ...
- 21
0
votes
1
answer
24
views
XGBoost regression scale invariant? 0 depth trees for target variable with small (1E-7) values
I thought the consensus was that XGBoost was largely scale-invariant and scaling of features isn't really necessary but something's going wrong and I don't understand what.
I have a range of features ...
- 103
2
votes
1
answer
29
views
Learning to Rank with Unlabelled Dataset
I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
- 51
0
votes
0
answers
17
views
Searching machine learning algorithm for regression problem with many features
I have a machine learning problem with about 160 features and 400 cases and I want to find the best predictors for a continuous outcome. The dataset contains variables of psychotherapists and clients. ...
0
votes
1
answer
50
views
XGBOOST with target column has categorical data and features also has categorical data
I have a huge dataset with the categorical columns in features and also my target variable is categorical.
All the values are not ordinal so I think it is best to use one hot encoding.
But I have one ...
0
votes
1
answer
15
views
Which metrics is used in the *training* of XGBoost : is it the one in the so-called parameter "eval_metric"?
In XGBoost, when calling the train function, I can provide multiple metrics, for example : 'eval_metric':['auc','logloss']
Which ones are used in the training and how to state it technically in the ...
1
vote
0
answers
18
views
Predictions using calibrated classifer
I find myself asking alot of calibration related questions recently - but i cannot find adequate material on it!
I am training a binary classifier to predict default. This probability will be used in ...
- 466
1
vote
0
answers
30
views
Are predictive features with 0 SHAP values included in the model?
I have trained and XGBoost by enforcing no-feaure interaction and calculated Global Shap values:
It looks like only 6 features have some SHAP values, whilst the remaining ones have a SHAP value of 0.
...
0
votes
0
answers
49
views
Why are SHAP values not an indication of cause?
I have trained an XGBoost Classifier and I am now trying to explain how and, most importantly, why the model has made the predictions it's made.
In the documentation entry Be careful when interpreting ...
0
votes
0
answers
15
views
Do monotonic constraints prevent an XGboost to capture non-linear relationships in the data?
I have trained an XGBoost model (for a binary classification problem) and I have tested two scenarios:
Scenario 1 - No Monotonic Constrained applied
In this case I get a Gini on the training sample of ...
0
votes
0
answers
15
views
Partial Dependence Plot - values range
I have trained an XGBClassifier on a dataset with binary target (0,1).
I have taken a look at the Partial Dependence Plot for each predictive characteristic.
For example:
Is it correct to assume ...
1
vote
0
answers
37
views
Differences between Feature Importance and SHAP variable importance graph
I have run an XGBClassifier using the following fields:
...
1
vote
1
answer
19
views
why my boosting model overfits even with just 4 features out of 61?
I am working on a binary classification problem using balanced bagging random forest, neural networks and boosting techniques. my dataset size is 977 and class proportion is 77:23.
I had 61 features ...
- 2,129
1
vote
0
answers
17
views
What is the formula of gradient boosting trees model?
I have been reading about gradient boosting trees (GBT) in some machine learning books and papers, but the references seem to only describe the training algorithms of GBT, but they do not describe the ...
- 66
0
votes
0
answers
39
views
lapse model and xgboost
A friend of mine has created a model for probability to lapse at one year of a customer (target variable is a binary "lapsed" 1 yes or 0 no (target "will the customer lapse in the next ...
- 19
0
votes
0
answers
7
views
Extending a classifier with specialised features
Let's say we have an app and a classifier (GBDT) predicting whether a user is a good user or bad (whatever that means) based on generic signals that every user has like profile fields, how long they ...
- 101
0
votes
1
answer
73
views
Recommendations for tuning XGBoost Hyperparams?
XGBoost has quite a few hyperparameters to tune: max depth, min child weight, number of iterations, eta, gamma, percent of columns considered, and percent of samples considered.
It's computationally ...
- 101
1
vote
1
answer
61
views
Are there rules of thumb for xgboosts hyperperameter selection?
There are multiple parameters that need to be specified in the XGBClassifier. Certainly gridsearchcv could give some insight into optimal hyperparameters, but I would imagine there are some rules of ...
- 165
0
votes
1
answer
55
views
XGBoost Regression Prediction
I trained an XGBoost Regression model that tries to predict the number of conversions that a campaign provides. Independent variables are monthly dummy, location dummy, and 4 columns of campaign rules ...
- 3
0
votes
0
answers
197
views
Xgboost Multiclass evaluation Metrics
Im training an Xgb Multiclass problem, but im having doubts about my evaluation metrics,
heres my code + output
...
0
votes
0
answers
164
views
What is the difference between RandomForestClassifier and XGBRFClassifier?
What is the difference between RandomForestClassifier and XGBRFClassifier?
There is no detailed explanation about what XGBRFClassifier exactly is so I was wondering.
- 957
0
votes
1
answer
84
views
XGBoost model has features whose feature importance equal zero
I ran into this problem:
A XGBoost model(.pickle file , constrcuted under V0.7.post3) with 100 features in it ;
But I found 55 features in model (model.feature_importances_) show 0 feature importance
...
- 1
1
vote
1
answer
18
views
Why does an unimportant feature has a big impact on R2 in XGBoost?
I am training an XGBoost model, xgbr, using xgb.XGBRegressor() with 13 features and one numeric target. The R2 on the test set ...
- 21
0
votes
0
answers
80
views
Xgboost taking some time to run vs hyperopt
Sorry for long post,im triying to run a xgb model but for some reason takes like 20 to 30 min(per run) with a specific set of hyperparams, but when i run hyperopt to get best params, takes like 7 ...
1
vote
1
answer
29
views
Feature importance has more variables than included in .csv?
I have a .csv dataset with 26 variables, ranging from Age to Weight and so forth. I plotted a feature importance plot with;
...
- 13
6
votes
1
answer
225
views
Random Forest significantly outperforms XGBoost - problem or possible?
I have dataset of around 180k observations of 13 variables (mix of numerical and categorical features). It is binary classification problem, but classes are imbalanced (25:1 for negative ones). I ...
- 63
0
votes
1
answer
23
views
Binary classification with seperate training and testing datasets [closed]
I have two datasets (train.csv) and (test.csv) revolving around predicting the death outcome for a disease. Both sets include 20 independent variables (age, weight, etc), but only the train.csv ...
- 13
2
votes
3
answers
55
views
Example for Boosting
Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
- 163
1
vote
0
answers
20
views
Interpreting XGBoost's results (when they span between [0,0.5])
I would like to classify sentences into one of two different categories.
I trained a XGBoost model over a search grid with k-folds cross-validation. My data represents sentences, and features ...
- 31
0
votes
0
answers
15
views
XGBoost regressor with weird output value
A regression task that the y value’s range is around a * 10^-5 to b * 10^-4 and trying to use XGBoost to handle this question.
And the weird thing is, when finished model training, the model ...
- 155
0
votes
1
answer
136
views
What is The difference of xgboost.sklearn.XGBClassifier and xgboost.XGBClassifier?
xgboost.sklearn VS xgboost.XGBClassifier
Here is my code that I tried to train make_moons ...
- 43
4
votes
1
answer
165
views
XGBoost non-linear regression
Is it possible to use XGBoost regressor to do non-linear regressions?
I know of the objectives linear and logistic.
The ...
- 141
1
vote
1
answer
27
views
Reasons for a model predicting probability of being class 1 at x value
All,
This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
- 466
0
votes
0
answers
53
views
Is there a Softmax-like transformation with scale-invariance and linarity?
At the moment I'm using XGBoost to generate a prediction of probabilities with a custom objective-function to build something like an expert system. To do so I need to transform the raw XGBoost ...
- 11
0
votes
1
answer
34
views
1
vote
0
answers
16
views
What are the tradeoffs associated with the various XGBoost interfaces?
I'm spinning up with XGBoost today, and have already encountered at least two ways to train gradient boosted trees:
Approach 1 ("native" xgb)
source: XGB python intro
...
- 11
1
vote
1
answer
37
views
Feature engineering using XGBoost regressor [duplicate]
If I want to train a regression model through tree based algorithms like XGBoost. Suppose that there have 5 features x1, x2, x3, x4, x5 and a target y. And some experts said x2 minus x3 is highly ...
- 155
1
vote
1
answer
25
views
New classification in Machine Learning model with xgboost
I write a code in Rstudio with xgboost to solve a Machine Learning problem. This is my actual code:
...
- 411
1
vote
1
answer
152
views
How to make predictions on unseen data with different cardinality using xgboost
I am training an XGBoost regression model on a feature set $X$ that includes a feature $x_k$ with high cardinality (~100). First, I am using one-hot-encoding to convert $x_k$ and then split the set ...
- 21
2
votes
2
answers
163
views
Influence of imbalanced feature on prediction
I want to use XGB regression. the dataframe is coneptually similar to this table:
...
- 339