# which neural network topology to learn correlations between time series?

I have two (or more in principle) 1xN time series, and I would like to train a NN to predict the next value of both. I can arrange them as a 2xN matrix and feed a window from this matrix as input to the NN, but I'm not sure how to structure the NN itself.

I have made a NN with convolutions that can do a pretty decent job with a single series, but I'd like to exploit cross-series correlations. What topology works to let the NN notice correlations between the time series?

• CNNs handle 2D out of the box. In fact that is their biggest use case (images). Why do you think you have to use something fancier? Commented May 29, 2018 at 13:46
• What is the relationship between the two time series? If there is no relation, what is the reason for combining them as input? Commented Nov 22, 2018 at 15:13

## 3 Answers

It depends a little on what kind of correlations you're looking for. Are you expecting a correlation that is present at each time step/window, or a different level of correlation per time step/window? Are you doing a classification or a regression task? Sometimes predicting the next value involves classification, but I'll assume you're looking for regression for now.

As a starting point, try feeding each of those sequences separately as input to a recurrent neural network (start with a basic LSTM and pare it down if it's overkill). I like your suggestion of overlapping windows.

For example (pseudocode-ish):

series_1 = 1, 2, 3, ..., 100

series_2 = 5, 6, 7, ..., 200

input_1 = Input(series_1,       window_size)

input_2 = Input(series_2, window_size)

layer_1 = LSTM(input_1, input_2)

final_layer = fully_connected(layer_1)


You should definitely check an LSTM-RNN or GRU-RNN implementation; the second one is easier to understand and less computationally expensive.

A valuable example is:

https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/

You could explicitly provide correlation as an nn input, as calculated in Local Correlation Tracking in Time Series, with code available via Vlad's Blog.

I have also read a paper ( unfortunately I can't remember its title or names of authors for a google search ) where a prediction of future correlation is an explicit nn target as part of a multivariate target nn target. The authors claimed inclusion of correlation as a target improved the accuracy on the non-correlation part of the multivariate target.