Recurrent Neural Net (LSTM) batch size and input

Question

I am working in Keras to build LSTM models. I understand that setting STATEFUL=FALSE means that the different batches are treated as independent when training the model.

Suppose I want to build a model that uses 6 weeks of hourly temperature observations in order to predict tomorrow's 24 hourly temperatures. I have been using a batch size of 24, meaning each day of training data gets used independently by the LSTM when training. But this means that I am not allowing the RNN to pick up longer correlations across days, and only within days, correct? So should I instead be using a batch size of 6*7*24=168? Would this larger batch size allow for the LSTM to pick up longer range correlations across the 6 week training period?

Secondly, I have been using the previous day's observations $y_{(t-24)}$ as predictors for the current day's temperature (at the same time period) $y_t$. Is this a valid approach, or is this redundant with the long term correlations that the LSTM will be looking for?

Answer 1

Batch sizes are different to series length. Batch size refers to the number of observations, and are typically defined when one is performing mini batch stochastic gradient descent.

If you are doing N mini batch SGD, you need to ensure the inputs to your network are a N x 168 tensor. Batch sizes are normally 32-128 observations.

WRT your second question, the approach is fine, it's just different. See which works better.