# Catboost Categorical Features Handling Options (CTR settings)?

I am working with a dataset with large number of categorical features (>80%) predicting a continuous target variable (i.e. Regression). I have been reading quite a bit about ways to handle categorical features. And learned that one-hot encoding I have been using in past is really bad idea especially when it comes to lots of categorical features with many levels (read these post, and this).

While I've come across methods like target-based encoding (smoothing) of categorical features often based on mean of target values for each feature perhaps this post/kernel in Kaggle. Still I am struggling to find a more concrete way till I found CatBoost an open-source gradient boosting on decision trees released last year by Yandex group. They seem to offer extra statistical counting options for categorical features likely much more efficient than simple one-hot encoding or smoothing.

The problem is the documentation is not helpful how to set CTR settings. I have tried different ways but it just does not work. The doc says CTR setting as simple_ctr, to be given as (CTR setting section):

['CtrType[:TargetBorderCount=BorderCount][:TargetBorderType=BorderType][:CtrBorderCount=Count][:CtrBorderType=Type][:Prior=num_1/denum_1]..[:Prior=num_N/denum_N]',
'CtrType[:TargetBorderCount=BorderCount][:TargetBorderType=BorderType][:CtrBorderCount=Count][:CtrBorderType=Type][:Prior=num_1/denum_1]..[:Prior=num_N/denum_N]',
...]


Here is a super simple example, the data looks like this:

import pandas as pd
import catboost
data = [{'profit': '342','country': 'holland','account': 'Jones LLC', 'saving': 150, 'debt': -60, 'age': 28},
{'profit': '875','country': 'germany','account': 'Alpha Co',  'saving': 200, 'debt': -10, 'age': 42},
{'profit': '127','country': 'italy','account': 'Blue Inc',  'saving': 50,  'debt': -300,  'age': 38 }]
df = pd.DataFrame(data)


Here is a simple Catboost Regressor:

X_train = df.drop(['profit'],axis=1)
Y_train = df['profit']
categorical_features_indices = [0,2]

train_pool = catboost.Pool(X_train, Y_train, cat_features=categorical_features_indices)

model = catboost.CatBoostRegressor(
depth=3,
iterations=5,
eval_metric='RMSE',
simple_ctr=None)

model.fit(train_pool);


The simple_ctr, one of CTR settings, is the problem!! It is pity because it looks like it the package offers various methods, so far no way to access them.

UPDATE Aug. 9th, 2018: A few days ago I raised this problem to Catboost developer, see here, and they opened a ticket for it to provide a tutorial.

• Here is the description about catboost on categorical vaiable: tech.yandex.com/catboost/doc/dg/concepts/… – DiveIntoML May 28 '18 at 0:07
• Yes, I know the link; it gives a detailed theoretical description. Yet not enough description on how to use it is given; I mean practical usage instructions. I have have tried it a few months ago and it was not straightforward. – TwinPenguins May 28 '18 at 5:39
• I am currently only using the one_hot_max_size param and my understanding is that everything that is not one hot encoded, will be encoded using target mean expanding encoding. I am looking forward to the additional official documentation to understand the ctr settings better and how I can play with them. – Garima Jain Mar 13 '19 at 16:13

## 2 Answers

Did you try using the format provided like below:

['CtrType[:TargetBorderCount=BorderCount][:TargetBorderType=BorderType][:CtrBorderCount=Count][:CtrBorderType=Type][:Prior=num_1/denum_1]..[:Prior=num_N/denum_N]'

['BinarizedTargetMeanValue[:TargetBorderCount=1][:TargetBorderType=Uniform][:CtrBorderCount=5][:CtrBorderType=Uniform][:Prior=1]']

• Is this an answer? It feels more like a question/comment. – Stephen Rauch Aug 9 '18 at 20:34
• @Interested_Programmer: It does not work, try it yourself in the example I just provided. Have you tested it or you simply found it in the documentation? – TwinPenguins Aug 9 '18 at 21:13
• I apologize @StephenRauch. I was unable to run my notebook but found it usable in other functions to follow the format. In hindsight, I should have added it as a comment. I am also waiting for the creators to put out the new tutorial. – Interested_Programmer Aug 10 '18 at 21:06
• also this works. simple_ctr=['BinarizedTargetMeanValue']) – Interested_Programmer Aug 10 '18 at 21:20
• Thanks. But as you shown it only works as simple_ctr=['BinarizedTargetMeanValue'])! The other options can not be fed like [:TargetBorderCount=BorderCount] etc. If you know how to feed others together with CtrType, let me know. – TwinPenguins Aug 11 '18 at 7:21

I found out that in order to set the ctr parameters and all the components one should pass a list of strings, each string should contain the ctrType and one of its component:

• The first word of the string should be a ctrType for example Borders: (click here for catboost parameters)
• Then one component of the ctrType should follow. For example TargetBorderType=5.
• All together 'Borders:TargetBorderType=5'.
• Repeat the procedure to set an other component and add the new string to the list.

Example with two components set:

simple_ctr = ['Borders:TargetBorderType=Uniform', 'Borders:TargetBorderCount=50']