I'm trying to develop a model to predict a commodity price movement direction based on previous observations. The model should learn common technical analysis patterns, e.g. head and shoulders. So, I think I should use a stateful LSTM so that it maintain a long term state to keep track of technical analysis patterns.
On the other side, as the data set is updated daily, i.e. new observations are added, I need the model to keep learning and making predictions every day. So, in order to update the model parameters on each new observation, I think I should use
If I use the last N observations to predict the next M steps, the input tensor to the model would have the shape of
(1, N, num_features):
$$ X_1, X_2, ..., X_N \rightarrow Y_1, Y_2, ..., Y_M $$
Why should we
reset_statesafter training on train set and prior to predication on test set? I think we shouldn't as a TA pattern may be half in the train set and half in the test set. If we reset the states, the model can't recognize this pattern.
reset_statesand feed in data of a different commodity? as I only need the model to learn TA patterns not commodity specific characteristics. If so, how can I restore the
statesto make prediction on the first commodity?