I don't understand the difference in mechanics of a transformer vs LSTM for a sequence prediction problem. Here is what I have gathered so far:
suppose we want to predict the remaining tokens in the word 'deep' given the first token 'd'. Then the first input will be 'd', and the predicted output is 'e'. Now at the next time step, the previously predicted output 'e' is fed along with the previous hidden state which contains information on 'd'. This is done till the predicted output is
In the same example, how would the transformer work, and avoid sequential inputs? Would we be giving the entire word 'deep' as input and leave it to the network to get the characters in correct sequence, or do we only input 'd' (which is what we did in LSTM)?
I'm really confused and I think I am missing out on some very fundamental concepts here. Would be really thankful for your help. Thanks!