I am working with NLP and have character level embeddings.

I have embeddings learned from Wikipedia text.

Now, I want to learn embeddings from chat data (where misspellings and abbreviations are way more common). Usually, the character n doesn't follow from character b, however, during texting, this can be common because they are close together on the keyboard, and a misspelling occurs.

So, my questions is: what are strategies to incorporate character keyboard position information to a traditional character level embedding?

Note: it can be assumed that only QWERTY keyboards exist.


1 Answer 1


Character keyboard position information is an example of noisy channel model information, an error that depends on how a word is transmitted. It is very common to add noisy channel model information to spell checkers, including spell checkers that use character-level embeddings.

Most character-level embedding models would automatically learn to model common transmission mistakes. Characters that are frequently confused in the dataset would be embedded closer to each other because they frequently co-occur. There would be minimal gain by explicitly adding channel information to a character-level embedding model during training.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.