Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
278
questions
1
vote
0answers
22 views
I have tried 5 different types of model but all returns really low training accuracy (~64%) and low testing accuracy (~14%). What should I do?
I am working with a typical regressor problem. There are $6$ features in the dataset that I am concerned with. There are about $800$ data points in my dataset. The features and the predicted values ...
1
vote
2answers
25 views
Cost function - ideas
I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
0
votes
0answers
22 views
How to test model accuracy on new vs. historical data?
I created an XG Boost model to predict churn using a dataset of customers who were sold during 2018. The accuracy of the model is 89%. Does it make more sense to re-pull the 2018 dataset, where more ...
1
vote
0answers
17 views
Feature Importance Scores from Gradient Boosting vs Random Forest
In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier.
Would like to know what are the fundamental differences in how this attribute is ...
1
vote
1answer
19 views
Tweedie Loss for Keras
We are currently using XGBoost model with Tweedie loss for solving a regression problem which works very good, now I wanted to move our model to Keras and experience with neural networks, do anybody ...
0
votes
2answers
26 views
Xgboost Parameter Tuning
Are there methods to tune and train an xgboost model in an optimized time - when I tune paramaters and train the model it takes around 12 hours to execute?
I would ...
2
votes
1answer
16 views
How can I get an algorithm to have an evalutation metric based on aggregate predictions?
Let's say I have a model that makes a prediction per individual. An example data set is below. Normally, evaluation metrics (for example within the XGBoost algorthim), are used at the individual ...
1
vote
0answers
14 views
How does the feval parameter influences the XGBoost training process?
In the package XGBoost, is possible to modify the feval (evaluated function) to a personalized one (as shown in the link: MAPE eval metric).
I would like to know how is the training process of the ...
2
votes
1answer
11 views
Xgboost multiple class predictive performance beats one versus rest
I have an NLP task I'm tackling with xgboost (R implementation).
Before describing my doubt I'll give you some background:
I have a corpus of documents for which I did topic discovery, using a term ...
1
vote
1answer
23 views
Marketing Mix Model (MMM) using XGBoost?
Does anyone know of any literature on Marketing Mix Modelling (MMM) using XGBoost? Is this a viable techniques for MMM modelling? What would the advantages/disadvantages be?
I think it would be a ...
0
votes
0answers
18 views
XGBClassifier Feature Order
I'm working with XGBClassifier from the XGBoost Python API and recently found out that the class requires feature columns to be in the same order for predictions as they were used for training.
For ...
0
votes
3answers
31 views
How to interpret feature importance (XGBoost) in this case?
I found two dominant features from plot_importance. My dependent variable Y is customer retention (whether or not the customer will retain, 1=yes, 0=no). My ...
1
vote
1answer
24 views
Determining threshold in an area with very few samples of positive label
I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
0
votes
0answers
22 views
Using bagging and random forests together
I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of ...
0
votes
1answer
8 views
Product Prediction to group of customers
I have multiple groups of customer, say for segment 1 as shown in the pictures, I have a list of products that I can choose the cross-sell to that group.
Consider ...
1
vote
1answer
33 views
Xgboost in R: Dealing with extreme class imbalance
My data has an extreme class imbalance - 99.7% is 0's and 0.2% is 1's and almost all the predictor variables (6 out of 7) are categorical. I trained an xgboost classifier after performing an ...
3
votes
0answers
69 views
XGBoost Huge Dataset ~1TB
Can a gradient boosting solution like XGBoost or Lightbgm be used for a huge amount of data ? I have a csv file of 820GB containing 1 Billion observations and each observation has 650 datapoints.
Is ...
0
votes
1answer
22 views
XGBoost: Can the features in test data be a subset of the features used to train the model?
Is it a problem if the test data only has a subset of the features that are used to train the xgboost model? All my predictor variables (except 1) are factors, so one hot encoding is done before ...
0
votes
1answer
14 views
How to interpret a random variable in the variable importance?
I have a problem, for simplicity let's say it is a binary classification problem.
I am trying to solve this problem using XGBoost.
A standard output plot for any ML algorithm, is the feature ...
0
votes
0answers
28 views
Bad input shape; XGB
I'm trying out a simple code to test the xgboost library in python.
My input matrix has 17 features and 16,718 observations X = (16718,17)
My output has 3 ...
1
vote
1answer
37 views
correct ML approach
I wanted to get your thoughts on a problem I have been facing. I have daily level product sales information (about 4 years). The sales are affected by the typical factors such as seasonality, day of ...
0
votes
0answers
35 views
XGBoost not learning
I have developed a train set for XGBoost to apply a learning to rank function on top of with the following parameters:
...
2
votes
1answer
38 views
Is the number of iterations in gradient tree boosting just the number of trees?
I have been searching for a while and I just can't find any indication. When people talk about iterations in algorithms like XGBoost or LightGBM, or Catboost, do they mean how many decision trees i.e. ...
0
votes
1answer
42 views
GBM: small change in the trainset causes radical change in predictions
I have build a model using transactions data trying to predict the value of future transactions. The main algorithm is Gradient Boosting Machine. The overall accuracy on the testset is fine and there ...
0
votes
2answers
49 views
Can this problem be solved using deep learning?
I want to predict price of used cars. I have data like this:
Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.?
I used one hot approach for nominal features and ...
0
votes
1answer
260 views
What is out come cox regression in xgboost
I am running xgboost where objective is survival:cox and eval_metric is cox-nloglik. Y range from -800 to 800. However, predicted values are way to large in range from 10^3 to 10^13. I am not sure why ...
1
vote
1answer
13 views
xgboostclassifier prediction error after saving the model and restoring it
I have trained a xgboost model and during training, the prediction works fine. But if I stop the script and start a restoring script to restore and predict, then for the same test dataset I get every ...
1
vote
0answers
48 views
Variable Importance changes with oversampling
I am currently using Xgboost for a binary classification problem with highly imbalanced data in R.
I have used oversampling to train the model. This worked well, now however it comes to measuring ...
1
vote
0answers
65 views
Bayesian optimization for a Light GBM Model
I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
1
vote
0answers
213 views
Custom loss function for XGBoost
I am looking for code to implement a custom loss function instead of just classification error, or cross entropy for gradient boosted classification trees.
We are trying to model regime detection in ...
0
votes
1answer
28 views
XGBOOST CV() producing error
I am getting the following error while using xgboost.cv() (scikit-learn interface). I am working on a regression problem. Below is the code and trace. No idea why ...
2
votes
1answer
49 views
Training a model where each response in the observation data has a different known varience
I have a dataset where each response variable is the number of successes of N Bernoulli trials with N and p (the probability of success) being different for each observation. The goal is to train a ...
1
vote
2answers
101 views
Why does it not need to set test group when using 'rank:pairwise' in xgboost?
I'm new for learning-to-rank. I'm trying to learn the Learning to rank example provided by xgboost. I found that the core code is as follows in rank.py.
...
0
votes
0answers
28 views
Suggestion for model performance improvement for ML competition
I am working on highly imbalanced dataset and trying to increase accuracy(metric: roc_auc) of my model which is hovering around 82-83%. This is part of an internal ...
1
vote
1answer
212 views
Random forest vs. XGBoost vs. MLP Regressor for estimating claims costs
Context
I'm building a (toy) machine learning model estimate the cost of an insurance claim (injury related). Aim is to teach myself machine learning by doing. I have settled on three algorithms to ...
3
votes
1answer
75 views
Hyper parameters tuning XGBClassifier
I am working on a highly imbalanced dataset for a competition.
The training data shape is : (166573, 14)
...
4
votes
1answer
727 views
Lightgbm vs xgboost vs catboost
I've seen that in Kaggle competitions people are using lightgbms where they used to use xgboost. My question is: when would you rather use xgboost instead of lightgbm? What about catboost?
2
votes
0answers
185 views
Adjust class weights due to class imbalance and class importance Multi class classification XGBoost
With respect to this question and the answer given by @Esmailian, Would anyone be able to let me know if Class B has a higher importance or the positive class ( i.e. it needs to have a higher ...
1
vote
0answers
15 views
if new feature downgrade the score for xgboost what do I have to look at?
let say I'm predicting the housing price of Boston(kaggle).
if I got some score x
then I added new feature y_K
if this new feature drop the score.
what is wrong with this feature and what do I ...
0
votes
1answer
25 views
xgboost and linear regression new feature analysis
For linear regression, seems like a new feature has to be a linear relation with the target variable.
But If you make the new feature for the Xgboost, what do you have to consider to make a new ...
3
votes
1answer
269 views
XGBRegressor hyperparameter optimization using xgb cv function
I am trying to optimize hyper parameters of XGBRegressor using xgb's cv function and bayesian optimization (using hyperopt package). Here is the piece of code I am using for the cv part.
...
0
votes
1answer
32 views
correct setting of eval_set in multiclass classification xgboost python , error is “ Check failed: preds.size() == info.labels_.size()”
i have a multiclass classification problem with 3 classes [-1,0,1] . i'd like to use eval_set in xgboost. but it fails with error:
...
0
votes
0answers
26 views
Training xgboost model with more data having different characteristic
I have trained my model for ECG data which has 8528 ECG files having length 30s and sample rate 300 so total file length in csv ...
1
vote
1answer
63 views
0
votes
0answers
38 views
XGBoost multiclass class balancing using weight parameter [duplicate]
I have three classes in the target variable with representation ratios of,
class A:0.5
class B:0.3
...
0
votes
2answers
19 views
Given a single discrete data set, how should I divide it into training data and test data?
I have a dataset in libSVM format consisting of 6000 entries, each with 5 indices, and each index has a binary value 1 or 2. Each of the 6000 entries has a label of ...
0
votes
0answers
233 views
XGBOOST (sklearn interface) REGRESSION error
I am trying to run a GRIDSEARCHCV (sklearn) on XGBRegressor. Documentation on the parameter says that if regression, then objective = reg:squarederror.(see https://...
1
vote
0answers
42 views
Why am I getting accuracy of Xgboost model 0.00%?
I am trying to build Job Recommender System using Deep Learning.
dataset used
From this dataset i have taken only users.tsv, user_history.tsv, apps.tsv and jobs.tsv to build a hybrid recommender ...
0
votes
2answers
108 views
Discrete Ordinal Classification with Probabilities
If I have classes 1, 2, 3 and 4. But, I also need the probability for each of the other classes. I'm currently using XGBoost for one-vs-rest classification, but that means we're losing information ...
0
votes
0answers
33 views
Reverse engineering on Xgboost model
I am doing experiments on https://www.physionet.org/challenge/2017/sources/ submission.
I like one of the submission code, which use Xgboost to train the ...
