32
votes
Accepted
Lightgbm vs xgboost vs catboost
On Kaggle, LightGBM is indeed the "meta" base learner of almost all of the competitions that have structured datasets right now. This is mostly because of LightGBM's implementation; it doesn't do ...
- 1,954
8
votes
Are linear models better when dealing with too many features? If so, why?
There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use.
When you use ...
- 7,366
3
votes
Accepted
How to tell CatBoost which feature is categorical?
When you are training your Catboost classifier, you can pass the list of cat features like this in python :
CatboostClassifier has a parameter called Cat_Features which takes list of names and treat ...
- 1,904
3
votes
Accepted
Catboost not able to handle a very simple dataset?
"Traditional" tree models cannot extrapolate well outside the training data's range, so "I want to use shuffle = False for reasons beyond the scope ...
- 11.2k
2
votes
Why does Catboost outperform other boosting algorithms?
There is no free lunch among Catboost, XGBoost and LightGBM. In my experience, some cases I ...
- 73
2
votes
Accepted
Does Gradient Boosting perform n-ary splits where n > 2?
Gradient boosting can be applied to any base model, so doing it with a Quinlan-family decision tree (which allow for such higher-arity splits for categorical features) should make this possible. ...
- 11.2k
2
votes
Accepted
Difference between model score on test part and Kaggle public score
In general, you should expect to get lower scores on test sets than validation sets, since you took advantage of validation data to tune your model. But for a correctly trained model, the difference ...
- 111
1
vote
SMOTENC oversampling without one-hot encoding
I had this same scenario - I fixed it by converting the Category columns to 'object' instead of the 'category' type.
...
- 11
1
vote
If I use Weight of Evidence to transform categorical variables, do I still need to inform their indexes to Catboost
The goal to encoding is to transform categorical into numerical so that an algo can learn on them. So the general answer would be no, after encoding into numerical you shouldn't declare them as ...
- 2,567
1
vote
Accepted
Select threshold (cut-off point )for binary classification by desired fpr persentage value
I've done my research and testing and it's that simple:
...
- 131
1
vote
CatBoost solves the problem of bias in pointwise gradient estimates
If I understood correctly the pointwise gradient estimate is the empirical gradient given a point. And the bias come from the estimate is being heavily data dependent. I didn't read the full paper, ...
- 131
1
vote
How do we target-encode categorical features in multi class classification problems?
Feature hashing, such as category_encoders HashingEncoder() is widely applicable in such cases, with a controllable feature size/...
- 686
1
vote
Accepted
How does Catboost regressor deal with categorical features at predict time?
In a simplified way of putting it, we substitute the category id with the mean value of the training set target for this category. CatBoost implements some tricks like only using the preceding values ...
- 686
1
vote
Unable to tune hyperparameters for CatBoostRegressor
This seems to be an issue with Catboost, at least there is a (now closed) issue on GitHub. Probably open a new issue to let the developers know about this.
I tuned ...
- 7,366
1
vote
Catboost multiclassification evaluation metric: Kappa & WKappa
I posted the same question on the Catboost github (issues) page and got an answer.
The link can be found here: https://github.com/catboost/catboost/issues/1447
Answer:
Class weights and weights in ...
- 31
1
vote
Accepted
training gradient boosting algorithm in python testing in Golang
There's actually a few libraries that handle the inference part well. https://github.com/dmitryikh/leaves is probably the most common one and seems to fit your need.
- 2,400
1
vote
Does gradient boosting algorithm error always decrease faster and lower on training data?
I'd like to confirm that this situation is not really something to worry about and I do not overfit the data.
No, the situation is not worrying, you can consider it worrying when the test error ...
- 6,082
1
vote
Does gradient boosting algorithm error always decrease faster and lower on training data?
You should worry about overfitting when the test error rate starts to go up again. Until then I would set it aside. Overfitting is rather about the number of parameters, e.g. When two models with the ...
- 562
1
vote
How to achieve SHAP values for a CatBoost model in R?
catboost::catboost.get_feature_importance(model, pool = pool, type = "ShapValues")
- 11
Only top scored, non community-wiki answers of a minimum length are eligible