LSTM with return_sequences - "Training a model on multiple timesteps simultaneously"

Question

So I'm following Tensorflow's LSTM/time series tutorial and there's something I don't understand. I do understand what happens with return_sequences on/off, however, it is stated, that with return_sequences on you allow

Training a model on multiple timesteps simultaneously

I don't quite understand what this means. If you keep it off, the input will go through all the previous timesteps, and make a prediction. Your loss will be the error of this prediction vs. the actual. You're still training on all the timesteps, right (although not simultaneously I guess?).

Now if it's on, it's making a prediction on each timestep - what's the loss in this case? Is it the average error for each prediction in the sequence? Why is it better to switch return_sequences on, as it's done in the tutorial? Does the model learn more or faster in this case? How does it impact the learning?

It is also stated that

With return_sequences=True the model can be trained on 24h of data at a time.

Isn't that still the case with return_sequences=False? In the example they're still teaching the model to predict 1 timestep based on the previous 24.

Answer 1

Setting return_sequences to True or False depends entirely on which one is appropriate with respect to how you are going to make predictions at inference time:

• If you want your model to make a prediction only when it has consumed a whole sequence as input, then you use return_sequences=False, so that your loss is only based on the output after ingesting the whole sequence.
• If you want your model to be able to make predictions after any number of timesteps, you set return_sequences=True and set the loss over all the time step outputs. As we are dealing with time series, the appropriate loss could be the average squared error over all time step outputs.

About the statement "With return_sequences=True the model can be trained on 24h of data at a time", I think it means that you are training the model for each of the 24h time steps, instead of the single prediction trained with return_sequences=false.