# Predicting parallel time series with multiple features

I am trying to predict sales for 2 departmental stores which share similar demographic properties. My goal is to make a single LSTM model to predict sales from these parallel time series having multiple features.

My input features for training would be

+----------+-------+--------------+-------+
|   Date   | Store | DayOfTheWeek | Sales |
+----------+-------+--------------+-------+
| 1/1/2019 | A     |            2 |   100 |
| 1/2/2019 | A     |            3 |   200 |
| 1/3/2019 | A     |            4 |   150 |
| 1/1/2019 | B     |            2 |   300 |
| 1/2/2019 | B     |            3 |   550 |
| 1/3/2019 | B     |            4 |  1000 |
+----------+-------+--------------+-------+


and my output for training would be

+----------+-------+--------------+-------+
|   Date   | Store | DayOfTheWeek | Sales |
+----------+-------+--------------+-------+
| 1/4/2019 | A     |            5 |   220 |
| 1/4/2019 | B     |            5 |   700 |
+----------+-------+--------------+-------+


Problem is that LSTM takes input as 3D i.e (n_sample, n_timesteps, n_features) and I can pass a single time series for a specific store (e.g. A)

If I had a univariate mutiple time series I can reshape my input data as follows and pass it to LSTM.

+----------+---------+---------+
|   Date   | A_Sales | B_Sales |
+----------+---------+---------+
| 1/1/2019 |     100 |     300 |
| 1/2/2019 |     200 |     550 |
| 1/3/2019 |     150 |    1000 |
+----------+---------+---------+


But I need to identify how can I predict parallel multivariate time series? Is there any other way to define in Input LSTM layer that there are 2 time series with 2 features each i.e (2*2).

Here are two ways to model the problem. The first one is simpler, the second one is more complex but closer to your original statement of the problem.

# Store as an input feature

You can consider the store as a feature to pass to your LSTM. With two different stores, just add a binary input feature "store" where store A is 0 and store B is 1, for example.

Then, you can implement it this way.

from keras.models import Sequential
from keras.layers import LSTM, Dense

timesteps = 20
n_features = 5
model = Sequential()
# add +1 to n_features for the store identifier
model.compile(optimizer="rmsprop", loss="mse")


One sample of your data at a given timestep will be a vector of the form (feature_1, feature_2, ..., feature_n, store).

# Multivariate time-series prediction

Here we input both time series and aim to predict next values of both stores. So you have a shared-LSTM processing store separately, then concatentate both produced embeddings, and compute the predicted values.

from keras.models import Model
from keras.layers import LSTM, Dense, Concatenate, Input

timesteps = 20
n_features = 5

# Store A and B time-series inputs
a_inputs = Input(shape=(timesteps, n_features))
b_inputs = Input(shape=(timesteps, n_features))

# Stacked LSTM
lstm_1 = LSTM(32, return_sequences=True)
lstm_2 = LSTM(32)
# Stacked LSTM on Store A
a_embedding = lstm_1(a_inputs)
a_embedding = lstm_2(a_embedding)
# Stacked LSTM on Store B
b_embedding = lstm_1(b_inputs)
b_embedding = lstm_2(b_embedding)

# Concatenate embeddings and define model
outputs = Concatenate()([a_embedding, b_embedding])
outputs = Dense(64)(outputs)
outputs = Dense(2, activation="relu")(outputs)
model = Model([a_inputs, b_inputs], outputs)
model.compile(optimizer="rmsprop", loss="mse")

• Do you mean to one hot encode the column store? If yes, then in case of 100 stores there will be 100 column? Commented Jan 29, 2020 at 9:31
• Yes, you could equivalently one-hot encode the binary feature I called "store". In case of 100 stores, the complexity of your problem is exploding. It is similar to trying to predict stock prices of various assets at the same time. I guess you would have to simplify this by predicting the sales store by store, or by grouping stores and predicting their total sales (if it is still relevant to what you are interested in) Commented Jan 29, 2020 at 12:11
• I understand the idea of grouping the stores. Suppose there are 3 stores in a group, will the network be able to identify the time steps after adding stacking multiple time series? Correct me if I am wrong shouldn't the Dense layer be giving 2 outputs rather than 1 since we are predicting sales for 2 stores? Commented Jan 29, 2020 at 12:59
• I updated my answer: the first model considers the store as a "switch" for processing one store or another. The second model matches your suggestion, as the output is effectively the next sales values of each store. Commented Jan 30, 2020 at 5:57
• @A.B You are right, with this model, the LSTM will not learn any dependency between the two time-series, but this will be learned in the final fully-connected layers. Also sharing weights speeds up training and gives more training data to the LSTM. If you assume some knowledge on the dependency between store A and store B, then adding it somehow to the model would surely improve it. Commented Jan 20, 2021 at 12:55