0
$\begingroup$

I am trying to make a caption generator model. (Having problem with shapes) I am getting error as

Input to reshape is a tensor with 4096 values, but the requested shape requires a multiple of 6400


Help me out here .

here is the model


UNITS = 128
IMG_SIZE = 240
BATCH_SIZE = 32
IMG_SHAPE = (IMG_SIZE, IMG_SIZE, 3)
max_len = 50
VOCAB_SIZE = tokenizer.vocabulary_size()

def get_model():

    embdding_layer = Embedding(input_dim = VOCAB_SIZE, output_dim = UNITS, input_length = max_len, mask_zero = True)
    rnn = LSTM(UNITS, return_sequences=True, return_state=True)

    # image inputs
    image_input = Input(shape=IMG_SHAPE)
    print(image_input.shape)
    x = resnet_preprocessing(image_input)
    print('preprocess: ', x.shape)
    x = resnet(x)
    print('resnet: ',x.shape)

    x = Flatten()(x)
    print('Flatten:',x.shape)
    
    # x = layers.MaxPooling2D()(x)
    # print('pooling:',x.shape)

    x = Dense(UNITS)(x)
    print('dense: ',x.shape)

    # x = 
    x = tf.reshape(x, (-1, 50, 128))
    print('reshape: ',x.shape)
    print('')

    # text inputs
    text_input = Input(shape=(max_len,))
    print('text_input: ',text_input.shape)

    i = embdding_layer(text_input)
    print('embedding: ',i.shape)


    i, j, k = rnn(i)
    i, _, _ = rnn(i, initial_state=[j,k])
    print('i:', i.shape)

    #  attention between x and i
    l = Attention()([x, i])
    ll = Attention()([i, x])
    print('attentions: ',l.shape, ll.shape)
    
    #  concatnate x and i
    m = Concatenate()([l, ll ])
    print('concat attention: ',m.shape)
    
    m = Dense(VOCAB_SIZE)(m)
    print('dense out: ',m.shape)
    
    return keras.Model(inputs = [image_input, text_input], outputs = m)

output shapes

(None, 240, 240, 3)
preprocess:  (None, 240, 240, 3)
resnet:      (None, 8, 8, 2048)
Flatten:     (None, 131072)
dense:       (None, 128)
reshape:     (None, 50, 128)

text_input:     (None, 50)
embedding:      (None, 50, 128)
i:              (None, 50, 128)
attentions:     (None, 50, 128) (None, 50, 128)
concat attention:  (None, 50, 256)
dense out:      (None, 50, 19770)
$\endgroup$

2 Answers 2

2
$\begingroup$

The output of your dense layer is (None, 128), which for batch size of 32 is going to be 4096. The call to reshape says you want a tensor of (None, 50, 128) which is 6400, hence your error. Resnet preprocessing output according to https://www.tensorflow.org/api_docs/python/tf/keras/applications/resnet50/preprocess_input is 4d tensor (batch, three color channels), I'm guessing you want to do some convolution layers and pooling to downsample to (None, 50,128)? Also, having a max pooling layer (currently commented out) after a flatten is going to give you a single value, probably not what you want. I might suggest checking out the image classification example here: https://www.tensorflow.org/tutorials/images/classification.

hth

$\endgroup$
0
1
$\begingroup$

for anyone who is looking for answer. This is the change i made.Thanks to @brewmaster321 UPVOTE his answer

    # image inputs
    image_input = Input(shape=IMG_SHAPE)
    print(image_input.shape)

    x = resnet_preprocessing(image_input)
    print('preprocess: ', x.shape)
    x = resnet(x)
    print('resnet: ',x.shape)

    x = GlobalAveragePooling2D()(x)  # add a pooling layer\
    print('polling: ',x.shape)

    x = Dense(UNITS*max_len)(x)
    print('dense: ',x.shape) # now (32, 128*50) => (32, 6400(multiple of 6400)) as required

    x = tf.reshape(x, (-1, max_len, UNITS))
    print('reshape: ',x.shape)
    print('')

in Dense layer instead of just Dense(UNITS) now is Dense(UNITS*max_len)

removed Flatten() layer and added GlobalAveragePooling2D()

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.