0
$\begingroup$

I'm familiar with the common methods:

  • Label Encoding: {A, B, C} -> [0, 1, 2]
  • One-Hot Encoding: {A, B, C} -> [[1,0,0][0,1,0][0,0,1]]

What other encoding methods exist and when would I need to use them over the two common methods?

$\endgroup$
1
  • $\begingroup$ There is also Target encoding, but embedding sounds better\smarter. Which task do you want to solve using (un)supervised setup? $\endgroup$
    – Mario
    Dec 15, 2022 at 1:53

1 Answer 1

1
$\begingroup$

Embeddings:

A --> [vector_A] B --> [vector_B] ...

They are frequently used in neural networks to represent categorical values that can have a lot of different discrete values, like text. The vectors are obtained either in an ad-hoc previous training step or during the training of the model itself.

$\endgroup$
2
  • $\begingroup$ My understanding of embeddings is that they are lower dimensional transformations of higher-dimensional encodings e.g. a truncated PCA on a one-hot-encoded matrix. Is that right? $\endgroup$
    – njp
    Dec 13, 2022 at 22:41
  • 1
    $\begingroup$ Well, they don't necessarily need to be of lower dimensionality, they just are learned representations in a continuous representation space. Also, I don't know if the one-hot + PCA example is sound; normally, examples are trained so that "close" discrete values are close in the embedded space, for instance, skipgrams in text reflect the word co-occurrence statistics. $\endgroup$
    – noe
    Dec 14, 2022 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.