I have read the code of ELMo: https://github.com/allenai/bilm-tf
Based on my understanding, ELMo first init an word embedding matrix
A for all the word and then add LSTM
B, at end use the LSTM
B's outputs to predict each word's next word.
I am wondering why we can input each word in the vocab and get the final word representation from the word embedding matrix
A after training.
It seems that we lost the information of LSTM
Why the embedding can contains the information we want in the language model.
Why the training process can inject the information for a good word representation into the word embedding matrix