Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

170 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
33 views

Using a Subset of Categories in a Categorical Column

I have a XGBoost model and I'm going to retrain it by adding new features. There is a column in my data and it's about professions of the customers. It has 60 categories. I suppose there is no need to ...
1
vote
0answers
127 views

State of the Art/Research 2020 of Time Series Forecasting/Prediction

Im looking for the state of the art/research of time series data for forcasting/prediction. As far as im aware it is Extrem Gradient Boosting (XGBoost) or LSTM (neuronal networks) or are there other ...
1
vote
0answers
92 views

Gradient boosting Regression with zero-inflated outcome

I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary ...
1
vote
1answer
150 views

XGBoost multiclassification interpreting predicted probabilities

Let's consider an example. I have patient level data, their symptoms, reading from various medical tests. Based on that, I have built a binary classifier given patient data to classify if they are ...
1
vote
0answers
22 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
1
vote
1answer
47 views

scale_pos_weight using XGBoost's Learning API

I see it is possible to add a weight for unbalanced problems in XGBoost's Scikit-Learn API through scale_pos_weight. Does it have an equivalent in the Learning API? If not, is there a reason behind ...
1
vote
0answers
59 views

is there metric 'multi_logloss' for xgb crassifier?

lgb has the log_loss metric ...
1
vote
1answer
37 views

How to check for “statistical significance” of categorical feature in black box models

Let's say we have a categorical feature $X_i$ and we have build a black-box classification model like xgboost with $X_i$ as one of many predictors. We'd like to ask a question: does $X_i$ affects the ...
1
vote
0answers
37 views

Is copying parameter considered as plagarism?

So my friends and i are writing a kaggle assignment and the base code is written by me. One of my friend use my base code(feature engineering, labeling, etc.) and put it into a loop to find the best ...
1
vote
0answers
402 views
1
vote
0answers
33 views

Feature Vectors representation

I would like to know I how you represent a feature vector like this dataset wise. The vector length is dynamic but the each element has a fixed length (9). For xgboost implementation, do I just create ...
1
vote
0answers
64 views

Hyperparameter Tuning using Bayesian Techniques

I've been looking into Bayesian optimization for hyperparameter tuning and trying to compare the results I get to those I get using different methods (random grid search). I came across this site, ...
1
vote
1answer
77 views

xgboost in R have different results compared to boosted decision tree in Azure ML

I have a small data set (4000 records with 10 features) and I used XGBOOST in R as well as Boosted Decision Tree model in Azure ML studio. Unfortunately the results are different. I like to optimize ...
1
vote
0answers
73 views

Trying to beat random forest with xgboost

I have a small time series dataset of about 3000 samples and 5 features. With xgboost, my predictions seem biased (consistently overestimating the target). No matter how many estimators I throw at the ...
1
vote
1answer
58 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my model ...
1
vote
0answers
263 views

Use LightGBM or FFM - imbalanced dataset

I have a highly imabalanced dataset but one that is not sparse. In train there are 1328 positives out of 104000. In validation ...
1
vote
0answers
382 views

Improving recall in XGBoost algorithm

I have highly imbalanced dataset. I am using XGBoost and I got the following results without balancing the dataset out: Precision: 0.87 Recall: 0.79 F1: 0.83 My ...
1
vote
0answers
24 views

Is it possible to create nonbinary trees in XGBoost?

I'm looking through the documentation for XGBoost, and I'm not seeing any parameters relating to number of branches per node.
1
vote
0answers
53 views

Feature Importance Scores from Gradient Boosting vs Random Forest

In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier. Would like to know what are the fundamental differences in how this attribute is ...
1
vote
0answers
73 views

How does the feval parameter influences the XGBoost training process?

In the package XGBoost, is possible to modify the feval (evaluated function) to a personalized one (as shown in the link: MAPE eval metric). I would like to know how is the training process of the ...
1
vote
2answers
40 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
1
vote
1answer
72 views

GBM: small change in the trainset causes radical change in predictions

I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
1
vote
0answers
61 views

Variable Importance changes with oversampling

I am currently using Xgboost for a binary classification problem with highly imbalanced data in R. I have used oversampling to train the model. This worked well, now however it comes to measuring ...
1
vote
0answers
860 views

Bayesian optimization for a Light GBM Model

I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
1
vote
0answers
43 views

if new feature downgrade the score for xgboost what do I have to look at?

let say I'm predicting the housing price of Boston(kaggle). if I got some score x then I added new feature y_K if this new feature drop the score. what is wrong with this feature and what do I ...
1
vote
0answers
94 views

Why am I getting accuracy of Xgboost model 0.00%?

I am trying to build Job Recommender System using Deep Learning. dataset used From this dataset i have taken only users.tsv, user_history.tsv, apps.tsv and jobs.tsv to build a hybrid recommender ...
1
vote
0answers
98 views

Reverse engineering on Xgboost model

I am doing experiments on https://www.physionet.org/challenge/2017/sources/ submission. I like one of the submission code, which use Xgboost to train the ...
1
vote
0answers
85 views

Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20) 0 : 91% 1 : 9% To build model on this dataset having class imbalance, I have compared ...
1
vote
0answers
58 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
1
vote
1answer
57 views

Multiple XGBoost models or just 1 for a cetain type of category?

I am building a model to predict, say house prices. Within my data I have sales and rentals. The Y variable is the price of either the sales or rentals. I also have ...
1
vote
1answer
163 views

XGBoost becomes unstable when predicting more than ~300 classes

I'm using the Python implementation of XGBoosts (version 0.80) XGBoostClassifier to predict one of a large number of classes. My feature data consists of a sparse boolean matrix of ~10M rows, ~5k ...
1
vote
0answers
244 views

How does L1 Loss work in lightGBM

From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance. My question is that, when the objective is L1 loss/...
1
vote
0answers
26 views
1
vote
1answer
739 views

Using a combination of gradient boosting with LSTM for classification?

I am presently using an LSTM model to classify high dimensional tabular data which is not text/images (dimensions 21392x1970). I also tried XGBoost (Gradient boosting) in Python separately for the ...
1
vote
0answers
54 views

Using feature vectors from imagenet to train xgboost (vs a standard Conv net)?

I am planning to use feature vectors generated from imagenet to train an xgboost model. This is as opposed to training a standard convolutional network with the same image set. This is because we ...
1
vote
0answers
137 views

What would be the equivalent of R's mboost in Python?

I am looking for the Python equivalent of R's mboost package ( mboost ). Would that be xgboost?
1
vote
0answers
384 views

Comparing XGBR with CatBoost performance

I saw on a CatBoost site that it supposed to outperform any other boosted training model and decided to try it myself on a Kaggle's https://www.kaggle.com/c/house-prices-advanced-regression-techniques....
1
vote
0answers
347 views

xgboost gain vs kolmogorov smirnov

After running xgboost model with: objective = 'binary:logistic' eval_metric = 'logloss' I have a group of 3 variables that have the highest values of gain. Now, ...
1
vote
0answers
117 views

Regression Decision Tree - Normalize or Split into Ranges a continuos feature

I have in my dataset a feature named distances which ranges goes from 200 to 12000 (more or less). Since the other features have got values under 50 I need to do ...
1
vote
0answers
52 views

AdaBoost - Ensemble model perform poor than best weak classifier

Can Adaboost's ensemble classifier perform worse than the best of the weak learners considered? If so when in what case of weak learner the ensemble learning does not perform better?
1
vote
0answers
2k views

xgboost f-score

I'm using xgboost package on python 3.5 for time base predication The result of the f-score Partial dependence on the order of the columns in data frame. The rmse of the predication is a same. for ...
1
vote
0answers
98 views

separate decision tree for categorical feature values

Given either, different decision trees each based on a particular feature value (like separate models for each male and female) or a single decision tree, should both give the same result?
1
vote
0answers
214 views

How does XGBoost implement MAE loss?

As we all know, XGBoost constructs trees based on gradient. I wonder how does XGBoost define gradient of MAE loss, as MAE itself is not differentiable. After some digging of the source code, I found ...
1
vote
0answers
38 views

What gradient boosting library is the easiest to implement on mobile?

I think of implementing training and predicting in an app (both Android and iOS), but existing packages I found not seem to be very mobile-friendly (scikit-learn, xgboost, lightgbm). Random forest ...
1
vote
0answers
488 views

How to interpret continuous variables in a decision tree model?

After fitting a decision tree with some continuous variable, how do I interpret the effect that variable has on the target? For example I'm predicting target Y. From sklearn random forest or Xgboost ...
1
vote
0answers
472 views

xgboost - How do I treat document ID in pairwise ranking

I am trying to use xgboost in R for pairwise ranking for an implicit dataset. For simplicity, let's assume that I am dealing with a search problem, where I want to rank documents relative to a given ...
1
vote
0answers
738 views

XGBoost: predict on only valuable features

I have a somewhat strange case that I can't find an answer to anywhere. It is really only applicable if you have a legitimately large data set. Let me describe. XGBoost returns feature importance for ...
1
vote
0answers
66 views

Injecting random values as one input feature for feature selection results in a odd beaviour

I am trying to find a cutoff value, in the feature importance space to eliminate spurious features. So I am injecting a completely random generated feature (as one of the input features) and I cut the ...
1
vote
0answers
356 views

How to set subsample in lightgbm in R?

There is a subsample parameter for the XGBoost (xgb.train() function in R). What is the corresponding subsample parameter for ...
1
vote
0answers
729 views

Feeding data to Xgboost for recomender system

I am using xgboost for a recommender system. There are 3-4 recommended content on each page. My data consists of columns like page_id and advertisement_id. Currently for every page_id, there are 3-4 ...