Word2vec and Glove embeddings have the same vector representation for every word in the corpus and does not take context into consideration.
- The dog does bark at people
- The bark of the tree is hard.
In the above examples, Word2vec and Glove create one vector for the word "bark". But using Elmo, there would be two different representations for the word "bark" as it considers context. So, I am trying to the mechanism behind how Elmo gives a vector for a new sentence in the data set?
Say, we have a pre-trained model and it has different vectors for various words. When I have to use this model on a new sentence, does Elmo produce a new vector with completely new vectors? Then is the fine-tuning of the model always happening as it is applied to new data? If that is the case, when is it considered to be a completely trained model?