Multivariate time series forecasting with LSTM

I have the following problem:

$$\mathbf{Y}(t)$$ = real valued random vector of observations at time t, $$Y_i(t) \in R_{(0, 1)}$$

$$\mathbf{X}(t)$$ = real valued random vector of observations at time t, $$X_i(t) \in R$$

I want to forecast $$\mathbf{Y}(t+1)$$, given $$\mathbf{X}(t+1)$$ and $$\mathbf{Y}$$ history.

To sum up I'd like to forecast a vector of observations given its history and another vector of observations as the same timestamp.

To make it clearer: let's say that I want to forecast daily temperatures in London and Dublin, knowing the same day temperatures in Manchester and Liverpool and London and Dublin (and Manchester and Liverpool) historical data. It's a bit like a kalman filter where you want to estimate the state variables given some output.

How would you set up this problem in an LSTM?

• I still have a question, Ugur: why the window size u have mentioned above is lack of x(t), does it should be x(t- 9)... x(t-1)x(t+1) as a feature? – Dokka Aug 4 at 14:35

1)First and most important, do not give Yi(t) history as feature. You will just end in a model that replicates the previous input to minimize the error, a cheating model. For more detail, you can have a look at my explanations at two different questions:

https://stackoverflow.com/questions/52252442/how-to-handle-shift-in-forecasted-value/53141558#53141558

2)Create your labels by sliding your Y(t) one step forward so that your each sample will have a label of Y(t+1). That means you will delete sample #1 as a result.

3)Use a time-window for each sample for your features. Do not provide just your features as x(t) for the label y(t+1). For example, with a window size w = 10, provide x(t-9),x(t-8),.....,x(t-1),x(t+1) as a single sample for label y(t+1). Then you will be boosting the sequential nature of LSTM, possibly acquiring greater performance.

4)You can use Keras for your LSTM regression task, have a look at simple code piece from my old works, I modified it for your task:

nn = Sequential()
nn.add(LSTM(80, batch_input_shape=(64,11,20), return_sequences=True, recurrent_dropout = 0.1))
nn.add(LSTM(60, recurrent_dropout = 0.2))