Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Could you elaborate on this argument? I found the brief explanation from the docs unsatisfying:
stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch.
Also, when stateful = True is to be chosen? What are practical cases of its use?
This flag is used to have truncated back-propagation through time: the gradient is propagated through the hidden states of the LSTM across the time dimension in the batch and then, in the next batch, the last hidden states are used as input states for the LSTM.
This allows the LSTM to use longer context at training time while constraining the number of steps back for the gradient computation.
I know of two scenarios where this is common:
The training set is a list of sequences, potentially coming from a few documents (LM) or complete time series. During data preparation, the batches are created so that each sequence in a batch is the continuation of the sequence at the same position in the previous batch. This allows having document-level/long time series context when computing predictions.
In these cases, your data is longer than the sequence length dimension in the batch. This may be due to constraints in the available GPU memory (therefore limiting the maximum batch size) or by design due to any other reasons.
Update: Note that the stateful flag affects both training and inference time. If you disable it, you must ensure that at inference time each prediction gets the previous hidden state. For this, you can either create a new model with stateful=True and copy the parameters from the trained model with model.set_weights() or pass it manually. Due to this inconvenience, some people simply set stateful = True always and force the model not use the stored hidden state during training by invoking model.reset_states().
stateful=True does not make much sense in your case, because you are not providing batches that maintain the sequence continuity.
$\endgroup$
stateful = True to pass the previous hidden state to the computation of the next prediction. When you made stateful = False, you invalidated such an assumption. Therefore, all predictions are generated with an input hidden state of 0, effectively making the predictions unconditional from the previous text.
$\endgroup$
stateful = True, the training does not use the stored hidden because model.reset_states() is invoked just before every training iteration.
$\endgroup$
stateful = True + model.reset_states() idiom.
$\endgroup$