I am building an LSTM to attempt to learn the historic trend of some time-series data set (e.g. the daily share price of a company). When training my network, I am taking batches of size 1, each consisting of 25 sequential daily closing prices, where it then makes a prediction for the price on the 26th day.
Say the first sequence in an epoch used to train begins at $t=0$ and ends at $t=24$. I am using a stateful LSTM and hence taking the output state of one batch and inputting it to the state of the next batch, does this mean that my 2nd batch must be $[t=25, t=49]$?
Can I instead "slide" each batch by 1 time step so that the 2nd batch is $[t=1, t=25]$, or does this defeat the purpose of passing the state between batches?