4
$\begingroup$

I'm working on a sequence2sequence model with attention mechanism, thus I'm using an encoder-decoder architecture.

The decoder part code is :

# Now create the Decoder layers.
decoder_inputs = Input(shape=(len_target,))
decoder_emb = Embedding(input_dim=vocab_out_size, output_dim=embedding_dim)
decoder_lstm = LSTM(units=units, return_sequences=True, return_state=True)
decoder_lstm_out, _, _ = decoder_lstm(decoder_emb(decoder_inputs), 
               initial_state=encoder_states)
x2 = reduce_dim(decoder_lstm_out)
attn_layer = AttentionLayer(name='attention_layer')
attn_out, attn_states = attn_layer([z, x2])
decoder_concat_input = Concatenate(axis=-1, name='concat_layer')([x2, 
               attn_out])
lm= Lambda(decoder_concat_input)
decoder_d2 = Dense(vocab_out_size, activation="softmax")
decoder_out = decoder_d2(lm)
model = tf.keras.Model([encoder_inputs, decoder_inputs], decoder_out)

The reduce_dim function() is:

   def reduce_dim(decoder_lstm_out):
       x = decoder_lstm_out 
       x = tf.placeholder(tf.float32, shape=[None, 12, 12, 256])
       dim = tf.reduce_prod(tf.shape(x)[0:1])
       x2 = tf.reshape(x, [dim,12,256]) 
       return x2

An error is raised :

AttributeError: 'Lambda' object has no attribute 'shape'

At the line:

decoder_out = decoder_d2(lm)

If anyone can help me please?

Improve this question
$\endgroup$
5
  • $\begingroup$ I've not the time to check right now (so comment rather than answer), but shouldn't the Lambda layer be called on the previous output? That would explain why it complains about the Lambda "object" rather than a tensor or whatever. Actually, do you need the Lambda there at all? Would the Concatenate layer work if passed directly into decoder_d2? $\endgroup$
    – Ben Reiniger ♦
    May 3, 2019 at 3:30
  • $\begingroup$ If I pass the concatenate layer , it works in decoder_d2, but it raise a problem on the Model declaration , the error says: ValueError: Output tensors to a Model must be the output of a TensorFlow Layer (thus holding past layer metadata). Found: Tensor("dense_3/truediv:0", shape=(?, 12, 152), dtype=float32) $\endgroup$
    – Kahina
    May 3, 2019 at 22:06
  • $\begingroup$ I'm rather more used to keras than tensorflow directly, so I'm not sure. I've just seen some references that suggest using the lambda in this case, to properly wrap functions for use in the other framework; so maybe leave it in, but call it on the concatenation layer? $\endgroup$
    – Ben Reiniger ♦
    May 5, 2019 at 15:46
  • $\begingroup$ Even if I call it on the concatenation layer I get the same error. $\endgroup$
    – Kahina
    May 7, 2019 at 1:33
  • $\begingroup$ Sorry to just keep chasing things around, but in reduce_dim you assign to x twice in a row; is that what you're meaning to do? If you can provide a MWE, I could play around for a while myself rather than just suggesting things as I think of them. (Hopefully someone more familiar with tf and encoder-decoders will come along.) $\endgroup$
    – Ben Reiniger ♦
    May 7, 2019 at 1:46

1 Answer 1

Reset to default
1
$\begingroup$

The error happens on this line:

lm = Lambda(decoder_concat_input)

Keras is not able to infer the input shape to the layer so you need to specify the input_shape parameter explicitly. But it is also not clear why you are using a Lambda layer in the first place. The layer is supposed to wrap a function but decoder_concat_input is itself a Layer. That is probably an error.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.