Questions tagged [catboost]
The catboost tag has no usage guidance.
39
questions
0
votes
0
answers
5
views
Optuina pruning during CrossValidation, does it make sense?
I'm currently trying to build a model using CatBoost. For my parameter tuning, I'm using optuna and cross-validation and pruning the trial checking on the intermediate cross-validation scores. Here ...
0
votes
0
answers
19
views
SMOTENC oversampling without one-hot encoding
I'm using SMOTENC to oversample an imbalanced-dataset.
I thought the point of SMOTENC was to give the option to oversample categorical features without one-hot encoding them. The reason I don't want ...
- 1
0
votes
0
answers
11
views
Oddly classifier is more accurate than regressor for solving a regression problem - what could be happening?
I am working through a simple tabular supervised machine learning problem.
I have a continuous target variable y that is normalized to the interval 0-1 to represent ...
- 101
0
votes
0
answers
88
views
How to use early_stopping_rounds in the Final Model? (CatBoost example with Optuna)
Imagine we have a model in the sklearn pipeline:
...
- 1
0
votes
0
answers
18
views
In Catboost, how do I calculate the gradient and delta of the data entering the leaf node?
I am studying the following paper : https://arxiv.org/pdf/1706.09516.pdf
On page 7, the first paragraph reads:
At the candidate splits evaluation step, the leaf value ∆(i) for example i is obtained ...
- 1
0
votes
0
answers
147
views
TimeSeries forecasting with Catboost
After extensive research on both the documentation and internet itself, I found many articles showing how to fit() and predict() a CatboostRegressor, but all of them use split data for train/test (...
- 101
2
votes
0
answers
27
views
Catboost: Categorcial Feature Encoding
I would like to understand all the methods available in Catboost for
encoding categorical features.
Unfortunately, the published articles by Yandex
("CatBoost: gradient boosting with categorical ...
- 121
1
vote
1
answer
27
views
If I use Weight of Evidence to transform categorical variables, do I still need to inform their indexes to Catboost
I'm using Weight of Evidence (WOE) to encode my categorical features. Do I still need to inform Catboost that they are categorical features by using cat_features parameter?
- 131
1
vote
0
answers
28
views
Intuition behind catboost encoding techniques
Can anyone please help me in understanding the effect of various bucketing techniques used in CatBoost Algorithm for categorical features? Like there is border, buckets, binarized target mean, counter ...
1
vote
1
answer
96
views
Select threshold (cut-off point )for binary classification by desired fpr persentage value
I want to recreate catboost.utils.select_threshold(desc) method for CalibratedClassifierCV model.
In Catboost I can select ...
- 131
0
votes
0
answers
22
views
How to guarantee Catboost output being integers?
I am trying to use Catboost regressor to predict next integer of a sequence of integers. I am using 4 numbers as input to predict the fifth. My problem is, instead of integers, the model predicts ...
0
votes
1
answer
54
views
CatBoost solves the problem of bias in pointwise gradient estimates
I've been reading the following papers: https://arxiv.org/abs/1810.11363, https://arxiv.org/abs/1706.09516 and https://www.researchgate.net/publication/318030603_Fighting_biases_with_dynamic_boosting.
...
0
votes
0
answers
20
views
How to Predict Binary Classification problem having less dominant Features
I have following dataset.
Total 31 columns including Target.
Target column has value of either 1 or 0.
This is balanced dataset.
All 30 Feature columns also have value of either 1 or 0.
All these 30 ...
- 101
1
vote
1
answer
200
views
How do we target-encode categorical features in multi class classification problems?
Say I have a multiclass problem with a dataset as this:
...
- 440
0
votes
0
answers
19
views
How to handle tags/lists with CatBoost?
I have database like this:
Id, type, category1, category2, tags
1, ‘cosmetics’, 123, 456, [446, 354]
2, ‘electronics’, 234, 213, [55, 978, 12]
…
And I want to predict some value with ...
- 1
0
votes
1
answer
335
views
How does Catboost regressor deal with categorical features at predict time?
I understand that Catboost regressor uses target-based encoding to convert categorical features to numerical features when training. But how does Catboost deal with categorical features at predict ...
- 3
0
votes
0
answers
10
views
Problem with Duplicate Data in Case of CatBoost
We know that while converting categorical variables to numerics, CatBoost uses the following formula (source: documentation):
Now, suppose there are 2 duplicate data entries. Ideally, the value of ...
0
votes
0
answers
167
views
Using class weights with training on imbalanced dataset gives worse result w.r.t logloss than without weights
I am trying to make a model for usual binary classification that is able to predict probabilities of classes. I have not very big dataset of 10k objects where classes are imbalanced as 80:20 and tried ...
1
vote
0
answers
29
views
Catboost not working properly when I remove non important variables (source of randomness?)
I was wondering if anyone has encountered the same. The thing is, when I run a catboost boosting model, delete non important variables (feature importance by prediction importance = 0, in fact these ...
- 53
0
votes
0
answers
81
views
Analysis of prediction shift problem in gradient boosting
I was going through the Catboost paper section 4.1 where they talk about the 'Analysis of prediction shift' using an example consisting of 2 features which are bernoulli random variables. I am unable ...
1
vote
1
answer
1k
views
How to tell CatBoost which feature is categorical?
I am excited to learn that CatBoost can handle categorical features by itself. One of my features, Department ID, is categorical. However, it looks like numeric, since the values are like 1001, 1002, ....
- 75
1
vote
0
answers
131
views
How to tune a Catboost Regressor
I have been trying to study about hyperparameter tuning for CatBoost regressor for my regression problem. The only issue being I can't figure out what all parameters should I tune for my use case out ...
- 11
4
votes
1
answer
647
views
Are linear models better when dealing with too many features? If so, why?
I had to build a classification model in order to predict which what would be the user rating by using his/her review. (I was dealing with this dataset: Trip Advisor Hotel Reviews)
After some ...
- 161
0
votes
1
answer
188
views
Catboost not able to handle a very simple dataset?
This is a post from a newbie and so might be a really poor question based on lack of knowledge. Thank you kindly!
I'm using Catboost, which seems excellent, to fit a trivial dataset. The results are ...
- 127
1
vote
1
answer
74
views
Difference between model score on test part and Kaggle public score
I tested my CatBoostModel model on part of data and get 0.92 score, but Kaggle public score was 0.9. I found new hyperparameters via randomsearch, new model score was 0.925, but on Kaggle score fell ...
- 125
0
votes
0
answers
970
views
How to use the eval set in catboost appropriately?
Let's say you have a dataset, and you split it into 80% training and 20% testing. Naturally, you want to find the optimal hyperparameters for your model, so with the training set, you plan to do cross ...
1
vote
1
answer
455
views
Unable to tune hyperparameters for CatBoostRegressor
I am trying to fit a CatBoostRegressor to my model. When I perform K fold CV for the baseline model everything works fine. But when I use Optuna for hyperparameter tuning, it does something really ...
- 1,321
0
votes
1
answer
24
views
How do Classification Algorithms such as Catboost and Random Forest parse test data?
I would like to know how classification works with the algorithms listed above. My specific question is this, say I have a high signal continuous feature which has a certain distribution and I train a ...
- 1
0
votes
1
answer
185
views
Model Dump Parser (like XGBFI) for LightGBM and CatBoost
Currently my employer has multiple GLM in a live environment. I am interested in identifying new features and interactions to enhance the accuracy of these GLM; for now I am limited to the GLM ...
- 1,527
0
votes
1
answer
81
views
Why does Catboost outperform other boosting algorithms?
I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
- 33
1
vote
1
answer
155
views
Does Gradient Boosting perform n-ary splits where n > 2?
I wonder whether algorithms such as GBM, XGBoost, CatBoost, and LightGBM perform more than two splits at a node in the decision trees? Can a node be split into 3 or more branches instead of merely ...
- 221
1
vote
0
answers
138
views
Feature Selection before modeling with Boosting Trees
I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
- 11
2
votes
1
answer
518
views
Catboost multiclassification evaluation metric: Kappa & WKappa
I am working on an unbalanced classification problem and i want to use Kappa as my evaluation metric. Considering the classifier accepts weights (which i have given it), should i still be using ...
- 31
0
votes
1
answer
411
views
training gradient boosting algorithm in python testing in Golang
What are the best strategy to train and save a gradient boosting algorithm, e.g. LightGBM or XGboost or Catboost in Python but load the model in GoLang and make prediction with Golang ?
- 303
0
votes
0
answers
155
views
How to understand the definition of Greedy Target-based Statistics in the CatBoost paper
There is a method named Target statistics to deal with categorical features in the catboost paper. I still some confusion about the mathematical form. Could you some guys to expain how to compute it!
$...
- 247
3
votes
0
answers
67
views
What is the concept behind the categorical-encoding used in the CatBoost benchmark problems?
I'm working through CatBoost quality benchmark problems (here). I'm particularly intrigued by the methodology adopted to convert categorical features to numerical values as described in the ...
- 161
1
vote
2
answers
486
views
Does gradient boosting algorithm error always decrease faster and lower on training data?
I am building another XGBoost model and I'm really trying not to overfit the data. I split my data into train and test set and fit the model with early stopping based on the test-set error which ...
- 182
6
votes
1
answer
544
views
How to achieve SHAP values for a CatBoost model in R?
I'm asked to create a SHAP analysis in R but I cannot find it how to obtain it for a CatBoost model. I can get the SHAP values of an XGBoost model with
...
- 91
22
votes
1
answer
11k
views
Lightgbm vs xgboost vs catboost
I've seen that in Kaggle competitions people are using lightgbms where they used to use xgboost. My question is: when would you rather use xgboost instead of lightgbm? What about catboost?
- 5,921