# How to generate multiple captions from an image captioning model in Keras/Tensorflow

I am practicing one of the popular image captioning keras model (LINK IS HERE). Basically this model takes Flickr8k dataset where each image has 5 captions.

1000268201_693b08cb0e.jpg,A child in a pink dress is climbing up a set of stairs in an entry
way .
1000268201_693b08cb0e.jpg,A girl going into a wooden building .
1000268201_693b08cb0e.jpg,A little girl climbing into a wooden playhouse .
1000268201_693b08cb0e.jpg,A little girl climbing the stairs to her playhouse .
1000268201_693b08cb0e.jpg,A little girl in a pink dress going into a wooden cabin .


During training they feed all five captions with each image as below

for i in range(self.num_captions_per_image):
loss, acc = self._compute_caption_loss_and_acc(
img_embed, batch_seq[:, i, :], training=False
)


when I train this model and try to predict some images from the validation data it generates only one caption with each image sample. While I need to generate the same number of captions per image as we trained the model (i.e 5 captions per image).

I am confused which part of the below code should I change to generate the same number of images as my validation data have.

vocab = vectorization.get_vocabulary()
index_lookup = dict(zip(range(len(vocab)), vocab))
max_decoded_sentence_length = SEQ_LENGTH - 1
valid_images = list(valid_data.keys())

def generate_caption():
# Select a random image from the validation dataset
sample_img = np.random.choice(valid_images)

# Read the image from the disk
sample_img = decode_and_resize(sample_img)
img = sample_img.numpy().clip(0, 255).astype(np.uint8)
plt.imshow(img)
plt.show()

# Pass the image to the CNN
img = tf.expand_dims(sample_img, 0)
img = caption_model.cnn_model(img)

# Pass the image features to the Transformer encoder
encoded_img = caption_model.encoder(img, training=False)

# Generate the caption using the Transformer decoder
decoded_caption = "<start> "
for i in range(max_decoded_sentence_length):
tokenized_caption = vectorization([decoded_caption])[:, :-1]
predictions = caption_model.decoder(
)
sampled_token_index = np.argmax(predictions[0, i, :])
sampled_token = index_lookup[sampled_token_index]
if sampled_token == " <end>":
break
decoded_caption += " " + sampled_token

decoded_caption = decoded_caption.replace("<start> ", "")
decoded_caption = decoded_caption.replace(" <end>", "").strip()
print("Predicted Caption: ", decoded_caption)

# Check predictions for a few samples
generate_caption()


Predicted Caption:  a group of dogs race in the snow


The above code generates just one caption from the above image while in the actual dataset this image has 5 captions. I have made a rough image in paint to show what I really want.

Thank You