I'm familiar with the common methods:
- Label Encoding:
{A, B, C}->[0, 1, 2] - One-Hot Encoding:
{A, B, C}->[[1,0,0][0,1,0][0,0,1]]
What other encoding methods exist and when would I need to use them over the two common methods?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ There is also Target encoding, but embedding sounds better\smarter. Which task do you want to solve using (un)supervised setup? $\endgroup$– MarioDec 15, 2022 at 1:53
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Embeddings:
A --> [vector_A] B --> [vector_B] ...
They are frequently used in neural networks to represent categorical values that can have a lot of different discrete values, like text. The vectors are obtained either in an ad-hoc previous training step or during the training of the model itself.
-
$\begingroup$ My understanding of embeddings is that they are lower dimensional transformations of higher-dimensional encodings e.g. a truncated PCA on a one-hot-encoded matrix. Is that right? $\endgroup$– njpDec 13, 2022 at 22:41
-
1$\begingroup$ Well, they don't necessarily need to be of lower dimensionality, they just are learned representations in a continuous representation space. Also, I don't know if the one-hot + PCA example is sound; normally, examples are trained so that "close" discrete values are close in the embedded space, for instance, skipgrams in text reflect the word co-occurrence statistics. $\endgroup$– noeDec 14, 2022 at 8:47