Questions tagged [xgboost]
For questions related to the eXtreme Gradient Boosting algorithm.
604
questions
3
votes
1answer
24 views
XGBoost non-linear regression
Is it possible to use XGBoost regressor to do non-linear regressions?
I know of the objectives linear and logistic.
The ...
1
vote
1answer
16 views
Reasons for a model predicting probability of being class 1 at x value
All,
This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
0
votes
0answers
23 views
Is there a Softmax-like transformation with scale-invariance and linarity?
At the moment I'm using XGBoost to generate a prediction of probabilities with a custom objective-function to build something like an expert system. To do so I need to transform the raw XGBoost ...
0
votes
1answer
26 views
1
vote
0answers
11 views
What are the tradeoffs associated with the various XGBoost interfaces?
I'm spinning up with XGBoost today, and have already encountered at least two ways to train gradient boosted trees:
Approach 1 ("native" xgb)
source: XGB python intro
...
1
vote
1answer
26 views
Feature engineering using XGBoost regressor [duplicate]
If I want to train a regression model through tree based algorithms like XGBoost. Suppose that there have 5 features x1, x2, x3, x4, x5 and a target y. And some experts said x2 minus x3 is highly ...
1
vote
1answer
22 views
New classification in Machine Learning model with xgboost
I write a code in Rstudio with xgboost to solve a Machine Learning problem. This is my actual code:
...
1
vote
1answer
26 views
How to make predictions on unseen data with different cardinality using xgboost
I am training an XGBoost regression model on a feature set $X$ that includes a feature $x_k$ with high cardinality (~100). First, I am using one-hot-encoding to convert $x_k$ and then split the set ...
2
votes
2answers
146 views
Influence of imbalanced feature on prediction
I want to use XGB regression. the dataframe is coneptually similar to this table:
...
1
vote
1answer
24 views
0
votes
0answers
8 views
Imbalanced Classification: BOW vs doc2Vec in XGBoost with sample weights
I am new to machine learning. I have an imbalanced dataset of pages of reports with
class 1: 97%,
class 2: 2.2%
class 3: 0.25%
which are the different type of pages
I am mostly concerned with ...
0
votes
1answer
43 views
XGboost predict
I am trying to understand this XGboost example.
After training:
ptrain = bst.predict(dtrain, output_margin=True)
they make prediction on test data, but the problem ...
1
vote
1answer
19 views
Sklearn xgb.fit: TypeError: fit() missing 1 required positional argument: 'y'
I am new to ML, and XGB is really confusing me. I understand that for Python XGB can be imported directly from the xgb library or via SKLearn.
The methods for xgb from the direct xgb library also ...
3
votes
1answer
29 views
classification balanced target y [0,1] but imbalanced feature x [many 0 , few 1s] , maximize precision
I have a simple dataset with balanced target y (0 or 1) ,and imbalanced feature (many 0 , few 1's)
I aim to get high precision (don't care about recall)
I can get high precision of 0.53 if I just ...
1
vote
0answers
33 views
XGBoost in Python: How do I input the scale_pos_weight for multi-classes?
I have 3 classes and my weights are
Class 1: 1
Class 2: 0.333
Class 3: 0.167
How do I input them into the scale_pos_weight parameter?
I know for a binary classification, we input it like below:
...
0
votes
1answer
40 views
ColumnTransformer worse performance than sklearn pipeline
I have an (unbalanced , binary data) pipeline model consisting of two pipelines (preprocessing and the actual model). Now I wanted to include SimpleImputer into my ...
0
votes
0answers
47 views
How does XGBoost perform in Parallel
So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
1
vote
2answers
159 views
Python xgboost predicting future events
This is related to this article: https://towardsdatascience.com/forecasting-of-periodic-events-with-ml-5081db493c46
I found it interesting and tried to replicate it, having as a result a xgboost ...
0
votes
0answers
20 views
Different AUC values for xgb and sklearn built in functions [duplicate]
The model is trained with early stopping on a validation set:
...
0
votes
0answers
10 views
XGB Regression: Is there a way to handle somewhat bimodal Y variable?
I am using XGBRegression to predict on continuous percentage data with 80% of the values around 100, 10% around 0 and 10% data distributed in the middle. Models are struggling with predictions around ...
0
votes
0answers
60 views
Can XGBoost support vector outputs?
I am interested in fitting data (regression rather than classification) with individual targets which are vectors via an XGBoost type model. However, currently Python's xgboost.XGBRegressor model only ...
1
vote
0answers
17 views
XGBoost for a binary classification where features are of different types
I have a dataset of "questions in an exam" that contains features such as:
QuestionLength (float)
averageTimePerQuestion (float)
hasMedia (boolean, represented as 0 or 1)
averageOfHardWords ...
0
votes
0answers
24 views
Gradient boosting algorithms and filling categorical variables
I have house prices dataset Link on Kaggle and I am having some dilemma. Some categorical variables having explicit majority.
If we look at MSZoning and SaleType columns, there is "RL" type ...
0
votes
0answers
12 views
For a Multi-class Classification Problem, What are the Pros and Cons of using a Cascade ML model versus Single Multi-class Classification model?
I am developing an ML model for classification using tabular data. It has 5 classes right now and new classes are expected to be continuously added. (Already have a new one leading to an imbalanced ...
0
votes
0answers
15 views
What is the best way to create input data samples using in XGBoost for predicting number of next days that customer will come back to store
I'm building the tree-based model like a XGBoost to solve the problem about customer purchase cycle.
And I think, I will build 2 models which one is predicting the customer will come back to store in ...
2
votes
2answers
34 views
confused on "real score" vs "decision value" in classification trees
I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is:
A ...
1
vote
0answers
16 views
Classification for Ordinal labels - what tree-based methds can i use?
I have a label that has a natural ordering e.g. 0,1,2,3 where 0 is the worst activity measure and 3 is the best. For each label given by the model i need to also give the probability that it belongs ...
0
votes
0answers
16 views
lack of consistency in Bayesian optimization of xgboost's hyperparameters
I am trying to optimize the hyperparameters in an xgboost model using Bayesian optimization and the mlrmbo R package. The simplified code below seem to produce reasonable results, but the problem I ...
0
votes
0answers
16 views
User Churn Rate analysis - Binary classification
I have a dataset which has the logs of user clicks. This is a trail version(2 months) of the software. Users can use a special feature during this trail period to improve their sales. The number of ...
0
votes
0answers
9 views
Train and test data fixed during boosting?
I have question about boosting algorithm.
I know that boosting is a sequential process and it gives high weight to misclassification of previous model.
Then, its' train and test data are fixed through ...
1
vote
0answers
13 views
Why is the average prediction moving away from average response for a reg:gamma model
I'm predicting a response that I would typically model under a gamma distribution, with relatively simple paramters, I'm just using the default other than these:
learning_rate = 0.01
max_depth = 6
...
2
votes
1answer
41 views
Labeling and aggregating features issue
I am trying build a simple binary classifier (some tree based algorithm for now) and my training data will have features aggregated at the user level. So I'll have a unique records of each user. These ...
2
votes
2answers
154 views
Flask output not showing
I am trying to deploy a XGBClassifier model using flask. After giving the values to the relevant fields on the webpage, the ...
0
votes
0answers
18 views
Feature importance not providing score for each feature
I have a data frame of this shape (808616, 10744) and I use this model
...
0
votes
0answers
56 views
Should I use or tune `reg_lambda` or `reg_alpha` hyperparameters when using a tree booster in XGBoost
XGBoost has 3 types of boosters:
tree boosters (gbtree, dart)
linear booster (gbliner)
...
1
vote
1answer
39 views
How to interpret .get_booster().get_score(importance_type='weight') for XGBRegressor()
I am trying to do feature selection using XGRegressor(). I am doing this because I have many features to choose from over 4,000. Once I have a set of features I have a neural network I created to use ...
2
votes
1answer
54 views
Possible to use predict_proba without normalizing to 1?
I'm using xgboost multi-class classifier to predict a collection of things likely to fail. I want to run that prediction, and report anything that the classifier ...
0
votes
2answers
56 views
aggregation of feature importance
I have more of a conceptual question I was hoping to get some feedback on. I am trying to run a boosted regression ML model to identify a subset of important predictors for some clinical condition. ...
0
votes
1answer
150 views
Is feature importance in XGBoost or in any other tree based method reliable?
This question is quite long, if you know how feature importance to tree based methods works i suggest you to skip to text below the image.
Feature importance (FI) in tree based methods is given by ...
-1
votes
1answer
101 views
nested cross validation vs. train-test split
I am trying to understand the main benefits of conducting a nested cross-validation compared to a simpler train-test split. Let us say I would like to build a prediction model. I initially split my ...
0
votes
2answers
22 views
Concerns regarding small dataset with too many features
I have dataframe with 322 observations with 224 features. The observations has two classes, 0 or 1,which i'm trying to predict. class 0 has 168 observations and class 1 has 154 observations.
I was ...
0
votes
1answer
18 views
Gamma parameter in xgboost
As per the original paper on xgboost, the best split at a node is found by maximising the quantity below
$
\cal{L}_{\rm split} = \frac{1}{2} \sum \left [ \frac{G_L}{H_L + \lambda} + \frac{G_R}{H_R + \...
0
votes
0answers
10 views
Why is my validation score so much higher using TargetEncoder?
So I'm experimenting a bit with an XGBoost model & encoding the categorical variables using the target encoder from the category_encoders library. The code below shows how I split the dataset and ...
0
votes
0answers
17 views
How Random Forest and XGB 'Regressor' calculate feature importance
I am searching about how Random Forest and XGB 'regressor' calculate feature importance.
However, the most of discussion focus on Classifier.
I try to find out the answer in the official document but ...
0
votes
0answers
21 views
Is XGBoost active in the Gradient Boosting widget on the Orange platform?
When I open the Gradient Boosting widget xgboost is grayed out even though it is described in detail under the documentation section on Orange. The two options are Gradient Boosting (scikit-learn) and ...
0
votes
0answers
12 views
xgboost performance
XGBoostRegressor is not performing better than AdaBoostRegressor for the same set of parameters for some reason. Since my ...
2
votes
1answer
28 views
My own model trained on the full data is better than the best_estimator I get from GridSearchCV with refit=True?
I am using an XGBoost model to classify some data. I have cv splits (train, val) and a separate test set that I never use until the end.
I have used GridSearchCV to determine the best parameters and ...
0
votes
2answers
33 views
XgBoost given targets its only feature but fails when test targets are outside the range of training targets?
I'm learning to use XgBoost, and I'm doing an exercise involving predicting prices. However I'm noticing some weird behavior where XgBoost's predictions deviate from the target value even if I'm ...
2
votes
0answers
27 views
Static ML model or Time-Series? How to model/predict a binary target when I have time variant features but most features are constant?
I have been working with Real World data from patients. I have a dataset with information about 10million patients; Collected over a span of varying duration (5 to 20 years).
What I am predicting is ...
2
votes
1answer
22 views
What if root of a such tree is pruned in xgboost?
Extreme Gradient Boosting stops to grow a tree if $\gamma$ is greater than impurity reduction given as eq (7) (see below) , what does happen if tree's root has a negative impurity? I think there is no ...