Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Imbalanced classification

I've tried all kind of oversampling undersampling techniques and I've tried also weighted Xgboost ( the model I'm trying to improve) I couldn't surpass a very Bad F1 score : 0.09 What should I do
user avatar
0 votes
0 answers
14 views

Make fitted xgboost or linear regression model predicts values in thé future

I have a machine learning model that I fitted with xgboost and linear regression. My dataset has thirteen features and has price as the target. Is there any way to ...
user avatar
0 votes
0 answers
13 views

Can we use an independent t-test as a metric for feature importance?

I have a supervised binary classification problem. I tuned an xgboost model on the training set and achieved a reasonably high accuracy on the test set. Now I want to interpret the results of the ...
user avatar
  • 165
1 vote
0 answers
46 views

Does it make sense to use target encoding together with tree-based models?

I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model). Someone suggested to use target-encoding (mean/median of the ...
user avatar
  • 11
1 vote
1 answer
15 views

Random search grid not displaying scoring metric

I want to do a grid search of some few hyperparameters through a XGBClassifier of a binary class, but whenever i run it the score value (roc_auc) is not being ...
user avatar
0 votes
0 answers
7 views

Custom multi-label cross-entropy loss that boosts weight of particular errors

I am using XGBoost for a multi-label classification problem (objective is 'multi:softmax' in XGBoost). In my case there are 16 discrete output labels where only one is correct. However, depending on ...
user avatar
0 votes
0 answers
19 views

Decision tree from boosted tree regressor Google bigquery ML

If I set the num_parallel tree to 1 and max_iteration to 1 in boosted_tree_regressor of Google Big Query ML will it work as Decision tree regressor ? Also can such decision tree give negative ...
user avatar
0 votes
1 answer
17 views

ROC-AUC Imbalanced Data Score Interpretation

I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations. The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
user avatar
0 votes
1 answer
12 views

Meassage "AUC-PR: the dataset only contains pos or neg samples"

My goal is to fit column name 'My_Val' using columns 'B1','B2', I tried XGboost function as below. And "AUC-PR: the dataset only contains pos or neg samples" shows. I have no idea what went ...
user avatar
  • 105
0 votes
1 answer
23 views

How to provide Intentional Bias towards recent examples in Text Classification?

I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
user avatar
1 vote
1 answer
30 views

XGBoost results changing when one row is removed

I have a training dataset of 2,600 rows and 26 columns. I trained an XGBoost (1.3.1) Classification model using the data and evaluated it using a test set of c. 800 rows. Whilst experimenting I found ...
user avatar
0 votes
1 answer
24 views

XGBoost regression scale invariant? 0 depth trees for target variable with small (1E-7) values

I thought the consensus was that XGBoost was largely scale-invariant and scaling of features isn't really necessary but something's going wrong and I don't understand what. I have a range of features ...
user avatar
  • 103
2 votes
1 answer
29 views

Learning to Rank with Unlabelled Dataset

I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
user avatar
  • 51
0 votes
0 answers
17 views

Searching machine learning algorithm for regression problem with many features

I have a machine learning problem with about 160 features and 400 cases and I want to find the best predictors for a continuous outcome. The dataset contains variables of psychotherapists and clients. ...
user avatar
0 votes
1 answer
50 views

XGBOOST with target column has categorical data and features also has categorical data

I have a huge dataset with the categorical columns in features and also my target variable is categorical. All the values are not ordinal so I think it is best to use one hot encoding. But I have one ...
user avatar
0 votes
1 answer
15 views

Which metrics is used in the *training* of XGBoost : is it the one in the so-called parameter "eval_metric"?

In XGBoost, when calling the train function, I can provide multiple metrics, for example : 'eval_metric':['auc','logloss'] Which ones are used in the training and how to state it technically in the ...
user avatar
1 vote
0 answers
18 views

Predictions using calibrated classifer

I find myself asking alot of calibration related questions recently - but i cannot find adequate material on it! I am training a binary classifier to predict default. This probability will be used in ...
user avatar
  • 466
1 vote
0 answers
30 views

Are predictive features with 0 SHAP values included in the model?

I have trained and XGBoost by enforcing no-feaure interaction and calculated Global Shap values: It looks like only 6 features have some SHAP values, whilst the remaining ones have a SHAP value of 0. ...
user avatar
0 votes
0 answers
49 views

Why are SHAP values not an indication of cause?

I have trained an XGBoost Classifier and I am now trying to explain how and, most importantly, why the model has made the predictions it's made. In the documentation entry Be careful when interpreting ...
user avatar
0 votes
0 answers
15 views

Do monotonic constraints prevent an XGboost to capture non-linear relationships in the data?

I have trained an XGBoost model (for a binary classification problem) and I have tested two scenarios: Scenario 1 - No Monotonic Constrained applied In this case I get a Gini on the training sample of ...
user avatar
0 votes
0 answers
15 views

Partial Dependence Plot - values range

I have trained an XGBClassifier on a dataset with binary target (0,1). I have taken a look at the Partial Dependence Plot for each predictive characteristic. For example: Is it correct to assume ...
user avatar
1 vote
0 answers
37 views

Differences between Feature Importance and SHAP variable importance graph

I have run an XGBClassifier using the following fields: ...
user avatar
1 vote
1 answer
19 views

why my boosting model overfits even with just 4 features out of 61?

I am working on a binary classification problem using balanced bagging random forest, neural networks and boosting techniques. my dataset size is 977 and class proportion is 77:23. I had 61 features ...
user avatar
  • 2,129
1 vote
0 answers
17 views

What is the formula of gradient boosting trees model?

I have been reading about gradient boosting trees (GBT) in some machine learning books and papers, but the references seem to only describe the training algorithms of GBT, but they do not describe the ...
user avatar
0 votes
0 answers
39 views

lapse model and xgboost

A friend of mine has created a model for probability to lapse at one year of a customer (target variable is a binary "lapsed" 1 yes or 0 no (target "will the customer lapse in the next ...
user avatar
0 votes
0 answers
7 views

Extending a classifier with specialised features

Let's say we have an app and a classifier (GBDT) predicting whether a user is a good user or bad (whatever that means) based on generic signals that every user has like profile fields, how long they ...
user avatar
0 votes
1 answer
73 views

Recommendations for tuning XGBoost Hyperparams?

XGBoost has quite a few hyperparameters to tune: max depth, min child weight, number of iterations, eta, gamma, percent of columns considered, and percent of samples considered. It's computationally ...
user avatar
1 vote
1 answer
61 views

Are there rules of thumb for xgboosts hyperperameter selection?

There are multiple parameters that need to be specified in the XGBClassifier. Certainly gridsearchcv could give some insight into optimal hyperparameters, but I would imagine there are some rules of ...
user avatar
  • 165
0 votes
1 answer
55 views

XGBoost Regression Prediction

I trained an XGBoost Regression model that tries to predict the number of conversions that a campaign provides. Independent variables are monthly dummy, location dummy, and 4 columns of campaign rules ...
user avatar
0 votes
0 answers
197 views

Xgboost Multiclass evaluation Metrics

Im training an Xgb Multiclass problem, but im having doubts about my evaluation metrics, heres my code + output ...
user avatar
0 votes
0 answers
164 views

What is the difference between RandomForestClassifier and XGBRFClassifier?

What is the difference between RandomForestClassifier and XGBRFClassifier? There is no detailed explanation about what XGBRFClassifier exactly is so I was wondering.
user avatar
  • 957
0 votes
1 answer
84 views

XGBoost model has features whose feature importance equal zero

I ran into this problem: A XGBoost model(.pickle file , constrcuted under V0.7.post3) with 100 features in it ; But I found 55 features in model (model.feature_importances_) show 0 feature importance ...
user avatar
1 vote
1 answer
18 views

Why does an unimportant feature has a big impact on R2 in XGBoost?

I am training an XGBoost model, xgbr, using xgb.XGBRegressor() with 13 features and one numeric target. The R2 on the test set ...
user avatar
0 votes
0 answers
80 views

Xgboost taking some time to run vs hyperopt

Sorry for long post,im triying to run a xgb model but for some reason takes like 20 to 30 min(per run) with a specific set of hyperparams, but when i run hyperopt to get best params, takes like 7 ...
user avatar
1 vote
1 answer
29 views

Feature importance has more variables than included in .csv?

I have a .csv dataset with 26 variables, ranging from Age to Weight and so forth. I plotted a feature importance plot with; ...
user avatar
6 votes
1 answer
225 views

Random Forest significantly outperforms XGBoost - problem or possible?

I have dataset of around 180k observations of 13 variables (mix of numerical and categorical features). It is binary classification problem, but classes are imbalanced (25:1 for negative ones). I ...
user avatar
  • 63
0 votes
1 answer
23 views

Binary classification with seperate training and testing datasets [closed]

I have two datasets (train.csv) and (test.csv) revolving around predicting the death outcome for a disease. Both sets include 20 independent variables (age, weight, etc), but only the train.csv ...
user avatar
2 votes
3 answers
55 views

Example for Boosting

Can someone exactly tell me how does boosting as implemented by LightGBM or XGBoost work in real case scenerio. Like I know it splits tree leaf wise instead of level wise, which will contribute to ...
user avatar
  • 163
1 vote
0 answers
20 views

Interpreting XGBoost's results (when they span between [0,0.5])

I would like to classify sentences into one of two different categories. I trained a XGBoost model over a search grid with k-folds cross-validation. My data represents sentences, and features ...
user avatar
0 votes
0 answers
15 views

XGBoost regressor with weird output value

A regression task that the y value’s range is around a * 10^-5 to b * 10^-4 and trying to use XGBoost to handle this question. And the weird thing is, when finished model training, the model ...
user avatar
0 votes
1 answer
136 views

What is The difference of xgboost.sklearn.XGBClassifier and xgboost.XGBClassifier?

xgboost.sklearn VS xgboost.XGBClassifier Here is my code that I tried to train make_moons ...
user avatar
  • 43
4 votes
1 answer
165 views

XGBoost non-linear regression

Is it possible to use XGBoost regressor to do non-linear regressions? I know of the objectives linear and logistic. The ...
user avatar
1 vote
1 answer
27 views

Reasons for a model predicting probability of being class 1 at x value

All, This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
user avatar
  • 466
0 votes
0 answers
53 views

Is there a Softmax-like transformation with scale-invariance and linarity?

At the moment I'm using XGBoost to generate a prediction of probabilities with a custom objective-function to build something like an expert system. To do so I need to transform the raw XGBoost ...
user avatar
0 votes
1 answer
34 views

Should I add string feature columns?

If my dataframe looks like this: ...
user avatar
1 vote
0 answers
16 views

What are the tradeoffs associated with the various XGBoost interfaces?

I'm spinning up with XGBoost today, and have already encountered at least two ways to train gradient boosted trees: Approach 1 ("native" xgb) source: XGB python intro ...
user avatar
1 vote
1 answer
37 views

Feature engineering using XGBoost regressor [duplicate]

If I want to train a regression model through tree based algorithms like XGBoost. Suppose that there have 5 features x1, x2, x3, x4, x5 and a target y. And some experts said x2 minus x3 is highly ...
user avatar
1 vote
1 answer
25 views

New classification in Machine Learning model with xgboost

I write a code in Rstudio with xgboost to solve a Machine Learning problem. This is my actual code: ...
user avatar
  • 411
1 vote
1 answer
152 views

How to make predictions on unseen data with different cardinality using xgboost

I am training an XGBoost regression model on a feature set $X$ that includes a feature $x_k$ with high cardinality (~100). First, I am using one-hot-encoding to convert $x_k$ and then split the set ...
user avatar
2 votes
2 answers
163 views

Influence of imbalanced feature on prediction

I want to use XGB regression. the dataframe is coneptually similar to this table: ...
user avatar
  • 339

1
2 3 4 5
13