Catboost Categorical Features Handling Options (CTR settings)?

Question

I am working with a dataset with large number of categorical features (>80%) predicting a continuous target variable (i.e. Regression). I have been reading quite a bit about ways to handle categorical features. And learned that one-hot encoding I have been using in past is really bad idea especially when it comes to lots of categorical features with many levels (read these post, and this).

While I've come across methods like target-based encoding (smoothing) of categorical features often based on mean of target values for each feature perhaps this post/kernel in Kaggle. Still I am struggling to find a more concrete way till I found CatBoost an open-source gradient boosting on decision trees released last year by Yandex group. They seem to offer extra statistical counting options for categorical features likely much more efficient than simple one-hot encoding or smoothing.

The problem is the documentation is not helpful how to set CTR settings. I have tried different ways but it just does not work. The doc says CTR setting as simple_ctr, to be given as (CTR setting section):

['CtrType[:TargetBorderCount=BorderCount][:TargetBorderType=BorderType][:CtrBorderCount=Count][:CtrBorderType=Type][:Prior=num_1/denum_1]..[:Prior=num_N/denum_N]',
 'CtrType[:TargetBorderCount=BorderCount][:TargetBorderType=BorderType][:CtrBorderCount=Count][:CtrBorderType=Type][:Prior=num_1/denum_1]..[:Prior=num_N/denum_N]',
  ...]

Here is a super simple example, the data looks like this:

import pandas as pd
import catboost
data = [{'profit': '342','country': 'holland','account': 'Jones LLC', 'saving': 150, 'debt': -60, 'age': 28},
         {'profit': '875','country': 'germany','account': 'Alpha Co',  'saving': 200, 'debt': -10, 'age': 42},
         {'profit': '127','country': 'italy','account': 'Blue Inc',  'saving': 50,  'debt': -300,  'age': 38 }]
df = pd.DataFrame(data)

Here is a simple Catboost Regressor:

X_train = df.drop(['profit'],axis=1)
Y_train = df['profit']
categorical_features_indices = [0,2]

train_pool = catboost.Pool(X_train, Y_train, cat_features=categorical_features_indices)

model = catboost.CatBoostRegressor(
    depth=3,
    iterations=5,
    eval_metric='RMSE',
    simple_ctr=None)

model.fit(train_pool);

The simple_ctr, one of CTR settings, is the problem!! It is pity because it looks like it the package offers various methods, so far no way to access them.

UPDATE Aug. 9th, 2018: A few days ago I raised this problem to Catboost developer, see here, and they opened a ticket for it to provide a tutorial.

Answer 1

I found out that in order to set the ctr parameters and all the components one should pass a list of strings, each string should contain the ctrType and one of its component:

• The first word of the string should be a ctrType for example Borders: (click here for catboost parameters)
• Then one component of the ctrType should follow. For example TargetBorderType=5.
• All together 'Borders:TargetBorderType=5'.
• Repeat the procedure to set an other component and add the new string to the list.

Example with two components set:

simple_ctr = ['Borders:TargetBorderType=Uniform', 'Borders:TargetBorderCount=50']

Answer 2

Did you try using the format provided like below:

['CtrType[:TargetBorderCount=BorderCount][:TargetBorderType=BorderType][:CtrBorderCount=Count][:CtrBorderType=Type][:Prior=num_1/denum_1]..[:Prior=num_N/denum_N]'

['BinarizedTargetMeanValue[:TargetBorderCount=1][:TargetBorderType=Uniform][:CtrBorderCount=5][:CtrBorderType=Uniform][:Prior=1]']