I see examples of LSTM sequence to sequence generation models which use start and end tokens for each sequence.
I would like to understand when making predictions with this model, if I'd like to make predictions on an arbitrary sequence - is it required to include start and end tokens tokens in it?
It depends on what you use the LSTM for.
For sequence labeling or sequence classification, the special tokens are not necessary. Although, there might be a slight benefit of informing the network of what is the beginning and the end of a sentence, especially if the initial LSMT state is fixed and learned.
For autoregressive sequence-to-sequence models, the special tokens are crucial. The beginning-of-sentence token serves as an instruction to the decoder to start decoding (it needs a very first state to predict what the next first token is). The end-of-sentence token is an instruction for the decoding algorithm to stop generating more tokens.
Yes, if it's trained to have the start and end tokens than you will need to include them because otherwise it will incur a domain shift since the network in question is trained strictly on sequences that start and end tokens.
For the case of sequence generation, the end token is needed so that the prediction has the ability to "stop," as in, allowing itself to stop generating sequences. In general, most models have parts in the code that explicitly stop the autoregressive generation process once the LSTM outputs a "stop" token, so it's not optional.
