1
$\begingroup$

I'm implementing a sequence-2-sequence model with RNN-VAE architecture, and I use an attention mechanism. I have problem in the decoder part. I'm struggling with this error: IndexError: list index out of range When I run this code:

decoder_inputs = Input(shape=(len_target,))
decoder_emb = Embedding(input_dim=vocab_out_size, output_dim=embedding_dim)
decoder_lstm = LSTM(units=units, return_sequences=True, return_state=True)
decoder_lstm_out, _, _ = decoder_lstm(decoder_emb(decoder_inputs), 
             initial_state=encoder_states)

print("enc_outputs", encoder_outputs.shape) # ==> (?,256)
print("decoder_lstm_out", decoder_lstm_out.shape)# ==> (?,12,256)
print("zzzzzz", z.shape) # ==> (?,256)

attn_layer = AttentionLayer(name='attention_layer')
attn_out, attn_states = attn_layer([z,z], decoder_lstm_out)

The error is raised at the last line, and the traceback given:

Traceback (most recent call last):
     File "malek_tuto.py", line 197, in <module>
           attn_out, attn_states = attn_layer([z,z], decoder_lstm_out)
     File "C:\Users\lightland\Anaconda3\lib\site- 
       packages\tensorflow\python\keras\engine\base_layer.py", line 728, in 
       __call__    self.build(input_shapes)
     File "D:\PFE\Contribution\modele\layers\attention.py", line 24, in 
        build shape=tf.TensorShape((input_shape[0][3], input_shape[0][3])),
     File "C:\Users\lightland\Anaconda3\lib\site- 
       packages\tensorflow\python\framework\tensor_shape.py", line 615, in 
         __getitem__   return self._dims[key]
     IndexError: list index out of range

in AttentionLayer class , build function id defined by:

 def build(self, input_shape):

      assert isinstance(input_shape, list)
      print("hhhhhhhhhh",input_shape)
      print("jjknkjnjk")
      # Create a trainable weight variable for this layer.

       self.W_a = self.add_weight(name='W_a',
                               shape=tf.TensorShape((input_shape[0][2], 
                                input_shape[0][2])),
                               initializer='uniform',
                               trainable=True)
       self.U_a = self.add_weight(name='U_a',
                               shape=tf.TensorShape((input_shape[1][2], 
                               input_shape[0][2])),
                               initializer='uniform',
                               trainable=True)
       self.V_a = self.add_weight(name='V_a',
                               shape=tf.TensorShape((input_shape[0][2], 1)),
                               initializer='uniform',
                               trainable=True)

       super(AttentionLayer, self).build(input_shape)

If someone can help me I'll be so thankful, I cannot understant where the problem is, and how to resolve it.

Thanks in advance

$\endgroup$
  • $\begingroup$ In my case, input tensors didn't have a shape. That means, shape=(). I reshaped via: tf.reshape(x["time"], (1,)) $\endgroup$ – iedmrc Jul 23 at 13:23
0
$\begingroup$

Maybe you are not using the attention.py you think you are using. The error states that the problematic line in Attention build is:

shape=tf.TensorShape((input_shape[0][3], input_shape[0][3])),

but the code of attention.py you shared with us does not seem to contain such line. It is possible you also just forgot to save your changes to the new attention.py file.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.