Skip to main content
Share Your Experience: Take the 2024 Developer Survey
32 votes
Accepted

Lightgbm vs xgboost vs catboost

On Kaggle, LightGBM is indeed the "meta" base learner of almost all of the competitions that have structured datasets right now. This is mostly because of LightGBM's implementation; it doesn't do ...
aranglol's user avatar
  • 2,196
8 votes

Are linear models better when dealing with too many features? If so, why?

There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use. When you use ...
Peter's user avatar
  • 7,516
4 votes
Accepted

How to tell CatBoost which feature is categorical?

When you are training your Catboost classifier, you can pass the list of cat features like this in python : CatboostClassifier has a parameter called Cat_Features which takes list of names and treat ...
Ashwiniku918's user avatar
  • 1,994
3 votes
Accepted

Catboost not able to handle a very simple dataset?

"Traditional" tree models cannot extrapolate well outside the training data's range, so "I want to use shuffle = False for reasons beyond the scope ...
Ben Reiniger's user avatar
  • 11.9k
2 votes

Model Dump Parser (like XGBFI) for LightGBM and CatBoost

Both LightGBM and CatBoost have functions that give you the models' feature importances. They are described in the respective docs here: LightGBM: plot_importance and lightgbm.importance Catboost: ...
neal's user avatar
  • 231
2 votes

Why does Catboost outperform other boosting algorithms?

There is no free lunch among Catboost, XGBoost and LightGBM. In my experience, some cases I ...
zonna's user avatar
  • 73
2 votes
Accepted

Does Gradient Boosting perform n-ary splits where n > 2?

Gradient boosting can be applied to any base model, so doing it with a Quinlan-family decision tree (which allow for such higher-arity splits for categorical features) should make this possible. ...
Ben Reiniger's user avatar
  • 11.9k
2 votes
Accepted

Difference between model score on test part and Kaggle public score

In general, you should expect to get lower scores on test sets than validation sets, since you took advantage of validation data to tune your model. But for a correctly trained model, the difference ...
SimplyFarzad's user avatar
1 vote

SMOTENC oversampling without one-hot encoding

I had this same scenario - I fixed it by converting the Category columns to 'object' instead of the 'category' type. ...
Aidan Wade's user avatar
1 vote

If I use Weight of Evidence to transform categorical variables, do I still need to inform their indexes to Catboost

The goal to encoding is to transform categorical into numerical so that an algo can learn on them. So the general answer would be no, after encoding into numerical you shouldn't declare them as ...
Lucas Morin's user avatar
  • 2,234
1 vote
Accepted

Select threshold (cut-off point )for binary classification by desired fpr persentage value

I've done my research and testing and it's that simple: ...
Michael's user avatar
  • 131
1 vote

CatBoost solves the problem of bias in pointwise gradient estimates

If I understood correctly the pointwise gradient estimate is the empirical gradient given a point. And the bias come from the estimate is being heavily data dependent. I didn't read the full paper, ...
Allan's user avatar
  • 141
1 vote

How do we target-encode categorical features in multi class classification problems?

Feature hashing, such as category_encoders HashingEncoder() is widely applicable in such cases, with a controllable feature size/...
dx2-66's user avatar
  • 696
1 vote
Accepted

How does Catboost regressor deal with categorical features at predict time?

In a simplified way of putting it, we substitute the category id with the mean value of the training set target for this category. CatBoost implements some tricks like only using the preceding values ...
dx2-66's user avatar
  • 696
1 vote

Unable to tune hyperparameters for CatBoostRegressor

This seems to be an issue with Catboost, at least there is a (now closed) issue on GitHub. Probably open a new issue to let the developers know about this. I tuned ...
Peter's user avatar
  • 7,516
1 vote

Catboost multiclassification evaluation metric: Kappa & WKappa

I posted the same question on the Catboost github (issues) page and got an answer. The link can be found here: https://github.com/catboost/catboost/issues/1447 Answer: Class weights and weights in ...
Musa's user avatar
  • 31
1 vote
Accepted

training gradient boosting algorithm in python testing in Golang

There's actually a few libraries that handle the inference part well. https://github.com/dmitryikh/leaves is probably the most common one and seems to fit your need.
Tophat's user avatar
  • 2,430
1 vote

Does gradient boosting algorithm error always decrease faster and lower on training data?

I'd like to confirm that this situation is not really something to worry about and I do not overfit the data. No, the situation is not worrying, you can consider it worrying when the test error ...
Carlos Mougan's user avatar
1 vote

Does gradient boosting algorithm error always decrease faster and lower on training data?

You should worry about overfitting when the test error rate starts to go up again. Until then I would set it aside. Overfitting is rather about the number of parameters, e.g. When two models with the ...
N. Kiefer's user avatar
  • 572
1 vote

How to achieve SHAP values for a CatBoost model in R?

catboost::catboost.get_feature_importance(model, pool = pool, type = "ShapValues")
JensMB's user avatar
  • 11

Only top scored, non community-wiki answers of a minimum length are eligible