I m a researcher, a computational linguist and I m working on word embeddings, trying to understand them.
What are contextual embeddings and how could I start working and building on my own?
Thank you in advance.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Contextual word embeddings are a type of word representation that captures not only the semantic meaning of individual words, but also the context in which they appear. This is completely opposite to traditional word embeddings like Word2Vec or GloVe, which generate a single, context-independent vector for each word in the vocabulary.
For example, consider the word "bank" in the sentences "I sat on the bank of the river" and "I deposited money in the bank". Traditional word embeddings would generate the same vector for "bank" in both sentences, failing to capture the different meanings. Contextual embeddings, on the other hand, would generate different vectors for each instance of "bank", reflecting the different meanings implied by the context.
