Are RNN or LSTM appropriate Neural Networks approaches for multivariate time-series regression?

Question

Dear Data Science community,

For a small project, I've started working on Neural networks as a regression tool, but I am still confused about possibilities of some variants. Here's what I am aiming to do:

• I have multiple input data time series $X(t)=[X_1(t), X_2(t), X_3(t),X_4(t)]$, and multiple target data time series which I want to modelize $Y(t)=[Y_1(t), Y_2(t)]$. All data are available for training.
• I aim to train my model/regression on an interval $[t_0,t_n]$, and then be able to apply it on a larger different interval.
• I know that relation between my $Y$ and $X$ are non-linear, but also that in need to take in account lag, or inertia. For example, $Y_1(t)$ is dependant of $X_1([t-dt_1,t])$ and $X_2([t-dt_2,t])$. All $dt_n$ are different, but I have an approximate idea of how 'far' I need to reach.

With this in mind, through some research I have been guided to focus on Recurrent Neural Networks (RNN) and Long Short Term Memory (LSTM) networks. I aim to use TensorFlow/Keras to work on this. However, after some reading, I'm getting confused with those solutions. Many people present them in prediction applications, which supposedely means that those networks use data from a time interval (either $X$ or $Y$ in my case) on an interval $[t_0,t_n]$ to predict $Y(t_{n+1})$. But my objective is to use $X([t_0,t_n])$ to modelize $Y([t_0,t_n])$ (on the same time interval). I am getting confused with this notion of "prediction".

Therefore, are RNN and LSTM networks appropriate solutions for my multivariate time series regression/model project? Or am I already going the wrong way?

As a beginner in this field, any reference or link to ressources/tutorial, or demo, is also gladly welcome.

Answer 1

Here is a really good source to begin multivariate time-series forecasting in Keras using LSTMs.

I aim to train my model/regression on an interval $[t_0,t_n]$ and then be able to apply it on a larger different interval.

There's no harm in this as long as you perform the right kind of multi-step forecasting. If your problem requires you to train on $[t_0, t_n], \text{for some } n < 100$ and produce $y_1(t_{n+1}), y_2(t_{n+1})$ as outputs then APPLY it on $[t_n, t_n+100]$ then there will be issues in implementation, as most ML models will require you to provide the same shape as you did when you were training your model. Here's when sliding across time-series will help you. Simply put,

$$ \begin{align} \text{Train on: }& [t_0, t_n] &\text{ Output: }& y_1(t_{n+1}), y_2(t_{n+1}) \\ \text{Predict on: }& [t_1, t_{n+1}] &\text{ Output: }& y_1(t_{n+2}), y_2(t_{n+2})\\ \text{Predict on: }& [t_2, t_{n+2}] &\text{ Output: }& y_1(t_{n+3}), y_2(t_{n+3})\\ & \vdots & & \vdots\\ \end{align} $$

I know that relation between my Y and X are non-linear, but also that in need to take into account lag or inertia. For example, Y1(t) is dependent of X1([t−dt1,t]) and X2([t−dt2,t]). All dtn are different, but I have an approximate idea of how 'far' I need to reach.

Well, you can model your time-series data as $X_1(t-k),\cdots,X_1(t), X_2(t-d),\cdots,X_2(t)$ as $X$ (input) and $X_1(t+1), X_2(t+1)$ as $y$ (target), where $k$ and $d$ refer to different lags for each variable.

Answer 2

RNN are appropriate for modeling time series. Their recursive input provides possibliity to model dynamic systems basing of their time series. I would recommend create two different models for each output. I am personally using MATLABs NARX networks for making predictions models for insdustrial purposes and they do a thing. https://www.mathworks.com/help/deeplearning/modeling-and-prediction-with-narx-and-time-delay-networks.html;jsessionid=bb13c3b99f070979e406ab13ad33 here is a ref for how to use NARX networks for time-series prediction.
I do not know about doing it in TF/Keras but RNN should have good performance.