0
$\begingroup$

I understand that Catboost regressor uses target-based encoding to convert categorical features to numerical features when training. But how does Catboost deal with categorical features at predict time when the labels are completely unknown? How does an object at predict time go down the Catboost decision trees if the decision trees are expecting to see categorical feature values as numbers?

I tried looking at the official documentation but could only find when the encoding was done during training when the labels are available.

$\endgroup$

1 Answer 1

1
$\begingroup$

In a simplified way of putting it, we substitute the category id with the mean value of the training set target for this category. CatBoost implements some tricks like only using the preceding values when encoding the train set, but transforming the test set will use the whole train statistics anyway (https://github.com/catboost/catboost/issues/838).

What happens when a previously unseen category is encountered in the test set? According to https://towardsdatascience.com/categorical-features-parameters-in-catboost-4ebd1326bee5 unseen categories receive a value based upon prior (controlled by CTR arguments). In other words, same as https://catboost.ai/en/docs/concepts/algorithm-main-stages_cat-to-numberic with countInClass being zero. (category_encoders implementation of CatBoostEncoder() seems to just use the average train target value.)

$\endgroup$
1
  • $\begingroup$ Could you specifically prepare the model to better deal with unknown values for a given feature? That is, have some trees that don't rely on this feature at all and try to generalize on others. $\endgroup$
    – salmin
    Nov 7 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.