Tensorflow: How to pass multiple images to VGG16 layers

Question

This is a toy problem I am working on. I have an extruded N-sided polygon that I have rendered from 5 different randomly selected viewpoints. The classification task is to determine the number of sides of the polygon used. To do this I would like to feed the five generated images of each viewpoint into 5 parallel VGG16 models then take the block-4 features generated from each viewpoint and pool them for further processing.

To do this I generate 5 VGG16 models and concatenate their outputs. I'm using the Dense layer just for testing:

import tensorflow as tf
from tf.keras.applications.vgg16 import VGG16
from tf.keras.preprocessing import image
from tf.keras.layers import Dense
from tf.keras.applications.vgg16 import preprocess_input
from tf.keras.models import Model
import numpy as np


views = [VGG16(weights='imagenet', include_top=False, input_shape=(224,224,3)) for i in range(0,5)]

merged = tf.keras.layers.Concatenate(axis=1)([v.get_layer('block4_pool').output for v in views])
#Final Layer
output_layer = Dense(16, activation = "sigmoid", name = "output_layer")(merged)


model = Model(
    inputs=[tuple([v.input for v in views])], 
    outputs=[output_layer],
    name="merged"
)

Unfortunately this gives me the error:

ValueError: The name "block1_conv1" is used 5 times in the model. All layer names should be unique.

How can I use these models in parallel with the pre-trained weights?

Answer 1

The solution is to simply rename the layers of each VGG model, as follows:

views = [VGG16(weights='imagenet', include_top=False, input_shape=(224,224,3)) for i in range(0,5)]

# rename each VGG model
for i, vgg in enumerate(views):
    for layer in vgg.layers:
        layer._name = f'{layer.name}-{i}'

# NOTE: we include the index "i" to handle the renaming of layers
merged = tf.keras.layers.Concatenate(axis=1)([v.get_layer(f'block4_pool-{i}').output
                                              for i, v in enumerate(views)])
#Final Layer
output_layer = Dense(16, activation = "sigmoid", name = "output_layer")(merged)


model = Model(inputs=[tuple([v.input for v in views])],
              outputs=[output_layer], name="merged")

Answer 2

I was able to rename the layers and still use the existing saved weights using the following approach: https://nrasadi.medium.com/change-model-layer-name-in-tensorflow-keras-58771dd6bf1b