I am trying to implement https://dl.acm.org/doi/pdf/10.1145/3269206.3271794 .

Structure: enter image description here

As it said:

In particular, we integrate the embedding vectors learned from each individual recurrent encoder into a new conclusive embedding vector to jointly consider various time series patterns with different ⟨α, β⟩ configurations

For my understanding, it use multiple individual rnn cell to process different timeseries, then concat all hidden states together to form a 3D input which can use 2d conv extract features .

But I didn't see there is a way to create multiple rnn cells in same layer , do I misunderstand?? If not , could you please give me a guide or an example ?


2 Answers 2


Not exactly familiar with this model architecture but, I don't think it works the way you described it.

From a quick skim, a simplified overview of the model may be

  1. data input: $X$ multiple time series
  2. RNN: pass each time series $x$ through a recurrent neural network and grab its hidden state
  3. Joint embedding: Concat hidden states from each $x$ series into an embedding with $X$ channels
  4. Convnet

But I didn't see there is a way to create multiple rnn cells in same layer , do I misunderstand?? If not , could you please give me a guide or an example ?

Essentially, any rnn will have a number of cells equal to the number of time steps. As an example, in NLP that number of cells would be equal to the number of words in a sentence, where each sentence in your trainset must have the same length.

I suggest you familiarise yourself with sequence models to better understand the concept of this paper.

  • $\begingroup$ Sorry, I misuse the word cell , I thought multiple layers in same layer need be called cell . I also saw some class in Keras named XXXcell . What I asking is exactly the 1,2,3 steps . One input , multile RNN(horizontally) generate hidden state , concat to 2dconv . All examples I saw are stacked RNN (vertically ) . $\endgroup$
    – Mithril
    Nov 23, 2020 at 2:18
  • $\begingroup$ Not sure what you mean I am afraid, feel free to clarify further. You could stack more layers upon cells VERTICALLY. the number of cells HORIZONTALLY must be equal to the number of time steps and are not to parametarise. $\endgroup$
    – hH1sG0n3
    Nov 23, 2020 at 10:17

Use funcational api solve the problem .

Structure likes this :

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

data = pd.DataFrame(np.random.uniform(size=(1000,3)), columns=['Sales', 'SalesDiff7', 'SalesAggMean7'])

multi_inputs = []
multi_outputs = []
window_size = my_window.input_width

for i in range(data.shape[1]):
    ti = keras.Input(shape=(window_size, 1), name=f't{i}')
    tlstm = layers.LSTM(32)(ti)
r = tf.stack(multi_outputs, axis=-2) 
result = keras.layers.Dense(units=1)(fc)

model = Model(


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.