# Tag Info

## New answers tagged xgboost

0

The paper you refer to actually states the following intuition: Algorithm 1 estimates $E[f(X)|do(X_S = x_S )]$ by recursively following the decision path for $x$ if the split feature is in $S$, and taking the weighted average of both branches if the split feature is not in $S$. It seems to be a slight modification of the original description in arxiv ...

0

The solution to the above problem was to use XGBRegressor instead of XGBClassifier. Just swapping it in seems to have worked.

1

You have to fit the RandomizedSearchCV first in order to access this attribute. random_search.fit(X_train, y_train) print(random_search.best_estimator_)

4

Assuming your x axis is nrounds(Or ntrees) parameter, XGB is an ensemble of many many trees built on top of one another. Your XAxis indicates how many trees have been used. Consider 2 points at x = 100 and x= 200, When you had 100 trees the train and test loss were close to .15 and 0.26, but on building 100 more trees on top of this train loss reduced to 0....

6

What can I understand/interpret from the training & test loss graph? This checking out the quality of the model. If the train and test set loss decreasing according to the number of epoch in the same way(i.e plots should overlap each other), that means that model is good. Otherwise, we have a problem. In your graph plots fastly separates(from x-value ...

0

Yes, you can use scale_pos_weight in the native python API; it goes in the params dictionary. E.g., params = {'objective': 'binary:logistic', 'scale_pos_weight': 2.5} model = xgboost.train(params, dmat) https://xgboost.readthedocs.io/en/latest/parameter.html#parameters-for-tree-booster https://github.com/dmlc/xgboost/blob/master/demo/kaggle-...

0

$R^2$ is just a rescaling of mean squared error, the default loss function for LightGBM; so just run as usual. (You could use another builtin loss (MAE or Huber loss?) instead in order to penalize outliers less.)

0

No: the trees' results are added together to produce the final score, so combining two models would produce outputs roughly twice as large as desired. (Gradient boosted trees change the target labels being fitted by each tree, so the 101st tree has "reset" the targets when training.)

0

In R, it's an option of the cross validation function : xgb.cv See the documentation here : https://www.rdocumentation.org/packages/xgboost/versions/0.4-4/topics/xgb.cv

0

Depending on the parameters you used for your model, it may not be calibrated in probabilities. That is, your model output a score, that is helpfull to give a relative order between your instance, but the score may not reflect the real % chance of the output happening. Softmax, will at least garanty that your output are between 0 and 1 and sum to one. This ...

1

The underlying model will be a stepwise function. I don't see any garanty that it will work better (or worse) with the transformation, in the general case. This may be different depending on you variable (for binary variables you may want to work directly on the linear predictor). However, in practice, if you know there is an underlying link transformation,...

2

Single or multi label doesn't make the difference. Cross validation is only split methodology. It just divides records in your data set to separate train and test splits. Python wrapper implements scikit API, so it'll work with any of the selection methods. Metrics will work too. Just remember to one-hot encode your labels. Is using cross validation for ...

1

Let´s say that the best way to choose is empirical. You run both algorithms in the dataset and check which one has better performance. It's true that you can do a lot of theoretical analysis but at the end you have to try no matter what. They both use decision trees ensemble so the results should not be too different. By experience gradient boosting tends ...

0

Multiclassification with xgboost is normally done One vs All. Meaning that first you try to predict if it belongs to class one or not. Then two second class or not... You can understand as the probability of belongin to one class and not to the rest. Let's put an example for better understanding: say that you have a ton of patients that have 3 illnes: ...

3

In ranking applications of information retrieval, training data consists of queries and documents matching them together with relevance degree of each match. For example when searching in something in google, the training data may be prepared manually by human assessors (or raters, as Google calls them), who check results for some queries and determine ...

0

This is the main advantage of ML: if the variable has any predictive value (that is not included in another variable), it should be used by the model. So, generally speaking it doesn't really make sense to handpick your variables to make different versions of your model (however it make sense to handpick some you want to throw). That would just be equivalent ...

2

Depending on your choice of accuracy metric, you'll find that different balancing ratios give the optimum value of the metric. To see why this is true, consider optimizing precision alone vs. optimizing recall alone. Precision is optimized (=1.0) when there are no false positives. Upweighting negative data reduces the positive rate, and therefore the false ...

0

I might be a little bit late to the party, but I did exactly what you need in the follow-up challenge of CinC2017, where our algorithm gained the 2nd best score on the hidden test set. Our code is available at https://github.com/martinkropf/ecg-classification Paper is available at https://iopscience.iop.org/article/10.1088/1361-6579/aae13e

0

Instance Weight File XGBoost supports providing each instance an weight to differentiate the importance of instances. For example, if we provide an instance weight file for the "train.txt" file in the example as below: train.txt.weight 1 0.5 0.5 1 0.5 It means that XGBoost will emphasize more on the first and ...

Top 50 recent answers are included