Questions tagged [xgboost]

For questions related to the eXtreme Gradient Boosting algorithm.

160 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0
votes
0answers
39 views

What is the best way to train a gradient boosted model on a binomial dataset where the number of “observations” for each instance varies?

I have been trying to figure out the best way to train a gradient boosted model on a binomial dataset. To be more clear my dataset is in a format similar to this: [link to toy dataset]. https://i....
0
votes
0answers
13 views

using logistic:binary as objective in xgboost

i am using 'binary:logisitic' as my objective function in xgboost classifier. Therefore when i predict probabilities i.e: model.predict_proba(x_test)[:, 1] will ...
0
votes
0answers
25 views

how can i plot probability distribution of my classes in the way below?

All, I would like to plot the following: I have a binary classification problem where I am using xgboost as my 'model' below: ...
0
votes
0answers
20 views

Confidence score over xgboost logistic regression

The probabilities of logistic regression indicate how the certain the model is over predictions. if its 0.93 it means the model is 93% confident the label is 1 and 7% to be 0. or if the probability is ...
0
votes
0answers
11 views

XGBoost parameters that will overweight errors for a specific decile

My goal is to make predictions about the bottom decile of my dataset. If I can predict the bottom decile values well (ie. minimize the errors there) then I am not worried about making accurate ...
0
votes
1answer
42 views

XGBoost Log Loss different from GridSearchCV Log Loss

I have a classification problem where i am trying to predict if the data returns a 1 or 0. So you're classic binary classification. I have my set of data that I have split into the dependent variables ...
0
votes
0answers
7 views

Zero-inflated independent feature in tree-based models

What is the best approach to include a zero-inflated continuous independent feature (e.g., 90% of the values are Zero, 10% are >0) in a Tree-based models (DT, random forest, gradient boosting. etc). I ...
0
votes
0answers
29 views

XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?

We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
0
votes
0answers
12 views

Building Uplift Tree using boosting

I want to build an Uplift Model for multiple treatments. To get a good model, I would like to use boosting. How is it possible to use boosting with uplift modeling although we can't really calculate ...
0
votes
0answers
12 views

How Xgboost or Decision trees deal with discontinous data in tabular form for time series

I am participating in a kaggle competition by the name "predict future sales". There the objective is to predict sales for each shop for each item for next month. The data is given such that only ...
0
votes
1answer
22 views

Determine how each feature contribute to XGBoost Classification

so for a summary of what I have done: My dataset has 5 classes and 10 parameters. I used XGBclassifer from sklearn to investigate if I could use those 10 parameters to predict the class of each data ...
0
votes
0answers
15 views

Gamma objective function XGBoost

I am using XGBoost to predict a variable that is highly skewed and always is greater than zero. I did a significant search to see some materials for gamma objective function in XGBoost but I could not ...
0
votes
1answer
36 views

how does xgboost handle inf or -inf values?

all, i am using xgboost for binary classfication. I have infs and -infs in my data due to the fact i am calcaulting ratios from one col and and another e.g. df[col1]/df[col2] , since i have zeros ...
0
votes
0answers
59 views

How to implement custom loss function that has more parameters with XGBClassifier in scikit-learn?

I have following problem with implementing custom loss function with scikit-learn: I would like to implement Focal Loss as my objective function in XGBClassifier. However, I dont know how to pass ...
0
votes
0answers
11 views

News Recommendation Engine with XGBoost

I want to build a news recommendation engine with XGBoost, but the data I have contains implicit user ratings, view history of a user. I know what my X's will(user embeddings + Item Embeddings) be but ...
0
votes
0answers
11 views

expected y_pred to have dimension (1054,2) but got array with dimension (1054,)

I've used XGBoost algorithm here. The code is given below. ...
0
votes
0answers
12 views

Sagemaker - XGBOOST rank:ndcg

Does anyone know if the rank:ndcg is available on AWS Sagemaker? I am currently trying to run a model, but it seems like it's not implemented. Am I using an older xgboost version? Kinda new to AWS ...
0
votes
0answers
21 views

Predict best score on unlabelled test set

Data I have one dataset with $1500$ data points, each with $\sim 23 000$ features (gene expression data, if that matters). However, I've split this dataset into a labelled training set of size 1000, ...
0
votes
0answers
11 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
0
votes
0answers
71 views

ExtraTreeClassifier does not handle missing values

I am using sklearn.tree.ExtraTreeClassifier. It does not handle missing value in training data. All tree-based algorithms handle missing value internally. So, is ...
0
votes
0answers
90 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
0
votes
0answers
48 views

XGBoost Python API - performance by API, R vs Python inconsistencies, GPU memory and verbosity

I'm combining a few questions together as I feel that it could benefit others. XGBoost API Is there a performance boost (training time or accuracy) when you use the learning API vs the Scikit-Learn ...
0
votes
0answers
28 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
0
votes
0answers
13 views

Getting bad predictions for high true values of target variable

I am working on a counterfeit medicine sales prediction regression model. As the relationship between target & response variables is non-linear I used tree based regressors random forests and XGB. ...
0
votes
0answers
206 views

SHAP Kernel explainer for my pipeline model

I am trying to use SHAP kernel explainer to understand my XGBOOST model. My data is the lending club data and I am trying to predict the Grade of each customer. The data contains different types of ...
0
votes
0answers
79 views

Multi-output classifier using Tensorflow/Keras on tabular data

I have been tasked with combining a several classifier models we have into one model using deep learning (or something else). The reason for this is that, in future, it would be difficult to maintain ...
0
votes
0answers
61 views

Changing objective functions with penalty term in XGboost in R

I basically have something like: ...
0
votes
0answers
123 views

Examples of the use of xgboost for recommender systems?

Are there any state-of-the-art implementations of xgboost in recommender systems? I'm looking for GitHub implementations but also papers that discuss this. I've only found this paper https://...
0
votes
0answers
72 views

Prediction error without having a true value

Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
0
votes
0answers
35 views

Why XGBoost regressor predicts behavior but not the amplitude?

I am very new to machine learning and I am trying to use XGBoostRegressor for my machine learning model (it has to do with physical modeling). I found out that it works very well for predicting the ...
0
votes
0answers
33 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
0
votes
0answers
461 views

XGBoost for multi-label image classification

I am trying to use the xgboost classifier for a multi-label and multi-class image classification task. I have a list of images that can have up to 5 different labels in each of them. Before I use the ...
0
votes
0answers
29 views

Feature engineering ideas with dates, coordinates and other variables

I'm working on an ETA problem where I'm trying to estimate a time of arrival for a delivery. I have coordinates of pickup/destination, time of pick , infos about the rider, some other variables that ...
0
votes
0answers
77 views

Suggestion for handling specific missing data

I have data, that describes distance from given location to nearest object (e.g. school, shop etc). Because of performance reasons I couldn't scrape the data about objects, that are futher away than 2....
0
votes
0answers
13 views

Improve the results of imbalanced multi-classification multi-lables data

I have 10k rows of multi-classification (x1..x27,y), size of dataframe is: 28*10k and its ...
0
votes
0answers
23 views

Data Transformation Tips for xgboost's XGBClassifier

I have this X_train and test distribution for the 4 features 'X', 'Y', 'TX' and 'TY'. I realize the range of the distribution is widely varying .. Can you suggest a good way to clean/ transform that ...
0
votes
0answers
29 views

if I got feature importance of xgboost/LightGBM what is next?

If I have feature importances of different variables in a xgboost/LightGBM model, how do I use this information? Is it better to just use the top n features and retrain the model? Does the feature ...
0
votes
0answers
20 views

What is the best approach to train a multi-category regression model?

What is the best approach to train a multi-category regression model (using XBoost decision trees ensemble)? What are the ups and downs of each one? For example, if I want to train a model to predict ...
0
votes
1answer
61 views

Predict items customers would buy in next order

I am working on a time series classification problem to identify what items customers would buy in their next order (customers orders different products every week). Let's say we have a customer who ...
0
votes
1answer
55 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my model ...
0
votes
0answers
10 views

How to encode features that encode regular values as well as special categorical values

I was recently playing around with the FICO explainable machine learning challenge dataset. In the dataset, there are a bunch of numerical features which have values values typically in the 0-100 ...
0
votes
1answer
397 views

Specifying number of threads using XGBoost.train

When using the xgboost.train() function, all the threads are used. I would like to use a specific amount. Unfortunately, this function does not accept the ...
0
votes
0answers
270 views

Save xboost model from R as Python pickle

I trying save R model to pickle, but I getting error Evaluation error: Unable to convert R object to Python type. ...
0
votes
2answers
66 views

Xgboost take k best predictions

I have a mission of classification with a lot of classes. I am comparing some ML algorithms for this case and I would like to try xgboost. I am writing in python and I am trying to get the best 3 ...
0
votes
0answers
25 views

records with perfect correlation to the answer. Drop or Keep?

I have about 1000 records (5 numeric, 5 categorical vars) and about 25 of them have something in 5-level categorical variable that just gives away answer. It's just too obvious and I'm not worried ...
0
votes
0answers
34 views

I have tried 5 different types of model but all returns really low training accuracy (~64%) and low testing accuracy (~14%). What should I do?

I am working with a typical regressor problem. There are $6$ features in the dataset that I am concerned with. There are about $800$ data points in my dataset. The features and the predicted values ...
0
votes
0answers
44 views

How to test model accuracy on new vs. historical data?

I created an XG Boost model to predict churn using a dataset of customers who were sold during 2018. The accuracy of the model is 89%. Does it make more sense to re-pull the 2018 dataset, where more ...
0
votes
1answer
17 views

Product Prediction to group of customers

I have multiple groups of customer, say for segment 1 as shown in the pictures, I have a list of products that I can choose the cross-sell to that group. Consider ...
0
votes
0answers
115 views

XGBoost not learning

I have developed a train set for XGBoost to apply a learning to rank function on top of with the following parameters: ...
0
votes
3answers
84 views

Can this problem be solved using deep learning?

I want to predict price of used cars. I have data like this: Is this problem suitable for deeplearning or Should I use XGBOOST, RandomForest etc.? I used one hot approach for nominal features and ...