Yes, it makes sense. Seq2seq models represent, in the RNN family, the best for multistep predictions. More classical RNNs, on the other side, are not that good for predicting long sequences.
If you need to implement a seq2seq model in TensorFlow 2.0 / Keras, each model follows the following structure:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, RepeatVector, Dense, TimeDistributed
from tensorflow.keras.activations import elu
# these are just made up hyperparameters, change them as you wish
hidden_size = 50
seq2seq = Sequential([
LSTM(hidden_size, input_shape = (input_sequence_length, no_vars)),
RepeatVector(prediction_length),
LSTM(hidden_size, return_sequences = True),
Dense(hidden_size, activation = elu)
TimeDistributed(Dense(1, activation = elu))
])
and then train it as usual with seq2seq.compile()
and seq2seq.fit()
.
If you want to stack more LSTM()
layers on top of each other, simply add them to the model I depicted above. Please keep in mind that LSTM models are very computationally expensice; without a good GPU even this "basic" model could be painfully long to train.
One modification I'd suggest, looking at your image, is to make the LSTM-encoder and -decoder parts of equal size and depth.
Alternatively, you can implement a more classical "Autoencoder-like" architecture, with LSTM()
layers for encoding and decoding, and Dense()
layers in the middle. However, seq2seq models are the most powerful at the moment.
To my knowledge, the only models more state-of-the-art than this are attention models. The problem is that they are so much state-of-the-art that TensorFlow/Keras doesn't have built-in layers for them, and you'd have to create your own custom layers (it's a pain). The only extensive implementation of attention models I found is from this blog post, but things are going to be very very complicated here. I didn't try to implement one yet.