I am using Keras to build a CNN model that takes two types of images as inputs (input1, input2) and produce one output. The model classifies the two inputs into a class, and the number of images in each type are different. (The number of images in the input1 is 1500 and in input2 is 4000).
This is my code:
in1 = Input(...)
x = Conv2D(...)(in1)
# rest of the model
out1 = Dense(...)(x)
in2 = Input(...)
x = Conv2D(...)(in2)
# rest of the model
out2 = Dense(...)(x)
concatenated_layer = concatenate([out1, out2]) # merge the outputs of the two models
output_layer = Dense(no_classes, activation='softmax', name='prediction')(concatenated_layer)
modal= Model(inputs=[in1, in2], outputs=[output_layer])
Image generator code:
input_imgen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
rotation_range=5.,
horizontal_flip = True)
test_imgen = ImageDataGenerator()
def generate_generator_multiple(generator,dir1, dir2, batch_size, img_height,img_width):
genX1 = generator.flow_from_directory(dir1,
target_size = (img_height,img_width),
class_mode = 'categorical',
batch_size = batch_size,
shuffle=False,
seed=7)
genX2 = generator.flow_from_directory(dir2,
target_size = (img_height,img_width),
class_mode = 'categorical',
batch_size = batch_size,
shuffle=False,
seed=7)
while True:
X1i = genX1.next()
X2i = genX2.next()
yield [X1i[0], X2i[0]], X2i[1] #Yield both images and their mutual label
inputgenerator=generate_generator_multiple(generator=input_imgen,
dir1=train_iris_data,
dir2=train_face_data,
batch_size=32,
img_height=224,
img_width=224)
testgenerator=generate_generator_multiple(generator=test_imgen,
dir1=valid_iris_data,
dir2=valid_face_data,
batch_size=1,
img_height=224,
img_width=224)
My questions are:
How can I determine the value of steps_per_epoch
and validation_steps
if the number of the images in each type are different?
In general, How to deal with the multi input Convolutional Neural Network that takes different number of images?