I am working on a problem that current depends on word level embeddings created using Word2Vec. I am researching new methods to apply to this model and one was a character level embedding. I have not found much information on it, and I don't imagine Word2Vec but at a character level would be effective. Is there any insight on giving vector representations to characters for an overall classification model?
It completely depends on what you're classifying.
Using character embeddings for semantic classification of sentences introduces unnecessary complexity, making the data harder to fit. Although using n-grams would help the model deal with word derivatives.
Classifying words based on their derivative would be a task that would require character embeddings.
If you're asking whether it would be useful to train a model to embed characters like you would with word2vec- then no. And in fact would probably yield bad results. We use embeddings to implicitly encode that two data points are close together and therefore should be treated more similar to the model. The letter 'd' shouldn't be semantically closer to 'e' than 'q'.
For the sake of completeness, fastText uses sub-word information (character level n-grams) when creating embeddings and then for classification with some interesting results:
- can give prediction for the unknown words that have some similarity with known ones
- with interesting results for modelling syntactic relations like 'went' - 'go' + 'give' = 'gave'