There are several popular word embeddings available (e.g., Fasttext and GloVe); In short, those embeddings are a tool to encode words along with a sensible notion of semantics attached to those words (i.e. words with similar sematics are nearly parallel).


Is there a similar notion of character embedding?

By 'character embedding' I understand an algorithm that allow us to encode characters in order to capture some syntactic similarity (i.e. similarity of character shapes or contexts).


1 Answer 1


Yes, absolutely.

First it's important to understand that word embeddings accurately represent the semantics of the word because they are trained on the context of the word, i.e the words close to the target word. This is just another application of the old principle of distributional semantics.

Characters embeddings are usually trained the same way, which means that the embedding vectors also represent the "usual neighbours" of a character. This can have various applications in string similarity, word tokenization, stylometry (representing an author's writing style), and probably more. For example, in languages with accentuated characters the embedding for é would be closely similar to the one for e; m and n would be closer than x and f .

  • $\begingroup$ Thank you so much for your interesting and very illuminating answer. Could you be so kind to share some references and resources about the topic syntactic embeddings (specially emphasizing some algorithms to construct such embeddings) ? $\endgroup$ May 16, 2022 at 21:16
  • 1
    $\begingroup$ @RamiroHum-Sah actually I don't know much myself about syntactic embeddings, it's different from character embeddings. From a quick search I found a few papers: this one was published at ACL, a top NLP conference, so it's probably good. There are others, but I don't know their value. $\endgroup$
    – Erwan
    May 16, 2022 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.