2
$\begingroup$

Say I have a multiclass problem with a dataset as this:


user_id  price   target
-------+--------+-----
1         30      apple
1         20      samsung
2         32      samsung
2         40      huawei
.
.

where I have a lot of users i.e One Hot Encoding (OHE) is not doable. Target-encoders such as CatBoost have achieved great results for target encoding categorical features. The issue is, I can only find CatBoost/target-encoder for binary classification/regression (which makes sense, in some way).

Right now I have overcommed the issue by target-encode each class in the target, by OHE the target (since it often has fewer categories), and then "target-encoded" the user_id for each OHE encoding e.g with these two steps:

  1. OHE the target
user_id  price   ohe_apple   ohe_samsung   ohe_huawei
-------+-------+-----------+-------------+------------
1         30       1             0             0
1         20       0             1             0
2         32       0             1             0
2         40       0             0             1
  1. target-encode user_id for each ohe_-column (the numbers are just made up for the illustration):

price  user_id_apple user_id_samsung  user_id_huawei
-----+--------------+----------------+---------------
30         0.5            0.6             0
20         0.5            0.6             0
32         0               1              0
40         0               0              1
.
.

I have achieved a notable increase of performance in e.g neural network with this approach (where OHE cannot fit into the GPU) but I wonder if there is some target-encoder for high-cardinality features (like user-id) in a multiclass classification, for e.g CatBoost?

$\endgroup$
4
  • $\begingroup$ Does not Catboost do a one vs all approaching multiclassification? $\endgroup$ Commented Jul 26, 2022 at 2:43
  • $\begingroup$ arxiv.org/abs/2105.13783 $\endgroup$ Commented Jul 26, 2022 at 2:46
  • 1
    $\begingroup$ @CarlosMougan just by looking at the title it is for regression - I'm trying to tackle a classifcation problem $\endgroup$
    – CutePoison
    Commented Jul 26, 2022 at 7:34
  • $\begingroup$ How does catboost multiclassification handle the problem? $\endgroup$ Commented Jul 26, 2022 at 18:27

1 Answer 1

1
$\begingroup$

Feature hashing, such as category_encoders HashingEncoder() is widely applicable in such cases, with a controllable feature size/information loss tradeoff.

category_encoders also supplies a PolynomialWrapper(), automating the extension of binary target encoders to multiclass (still using OHE on the target inside).

Edit: valid point, hashing is target agnostic, so to speak. It is a better option than OHE still, and category_encoders devs even recommend it in a case of a huge amount of classes.

$\endgroup$
1
  • 2
    $\begingroup$ The OP is asking about Target Encoder not HashingEncoder $\endgroup$ Commented Jul 26, 2022 at 2:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.