Do word embeddings help with out of vocab tokens?

Question

I am performing sentiment analysis on a custom dataset of text with Keras but am a little confused about word embeddings. I have been able to train an "Embedding" layer and have also learned to load existing weights from Glove but am still facing a few problems. The main one being that there are certain "negative" words I know of but that do not appear in the vocab. Because of this, when i try examples with word that do NOT appear in the vocab (like the word "rubbish") the network does not know that this contains a negative sentiment.

Is there a way to use Word2Vec / Glove / etc to pass in the word rubbish, find the similarity to the word garbage, and then pass that known word into the network instead? And if so is that handled by the "Embedding" layer or is it an additional step I need to perform during pre-processing?

Additionally, how can I handle misspelled words? For instance, how can i associate "rubbbbbish" with "rubbish"?

I am new to text classification and would really appreciate a bit of guidance!

Answer 1

By definition an out of vocabulary word (OOV) is a word which haven't been seen in the training data, so it's virtually impossible to know which other word it is similar to since this would have to be determined with some training data. In the example that you mention (synonym), what you could use is a resource such as WordNet which tells you which word it's similar to, then you can use the other word embedding.

Cases of misspelled words is a different story: you could pass your OOV words through string matching techniques, or maybe use a character-based NN which would recognize the word despite the misspelling.

Both cases could be part of a pre-processing stage.