I have a basic doubt. Kindly clarify this.
My doubt is, When we are using LSTM's, We pass the words sequentially and get some hidden representations.
Now transformers also does the same thing except non sequentially. But I have seen that the output of BERT based models can be used as word embeddings.
Why can't we use the output of LSTM also as a word embedding? I can find sentence similarity and all with LSTM also ?
For eg : If I have a sentence " is it very hot out there"
Now I will apply word2Vec and get dense representations and pass it to my LSTM model. The output of my LSTM can also be used as word embeddings as we do the same with BERT?
My understanding was that LSTM is used to identify dependencies between words and using that learned weights to perform classification/similar tasks.