I'm trying to train a Siamese Network model for a signatures dataset using Keras API and considering the loss only seems not bad. But ironically enough the model accuracy stuck at 0.5.
Model Loss:
Model Accuracy:
My model is kind of a deep model, here's its architecture:
input = Input((128, 128, 1))
x = BatchNormalization()(input)
x = Conv2D(16, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(32, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(64, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(128, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(256, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Flatten()(x)
x = BatchNormalization()(x)
x = Dense(10, activation="tanh", kernel_regularizer="l2")(x)
embedding_network = Model(input, x)
input_1 = Input((128, 128, 1))
input_2 = Input((128, 128, 1))
tower_1 = embedding_network(input_1)
tower_2 = embedding_network(input_2)
merge_layer = Lambda(euclidean_distance)([tower_1, tower_2])
normal_layer = BatchNormalization()(merge_layer)
output_layer = Dense(1, activation="sigmoid")(normal_layer)
siamese = Model(inputs=[input_1, input_2], outputs=output_layer)
The model takes grayscale 128x128 images of signatures in batches of 16 images using the ImageDataGenerator class having 800 training samples and 200 validation samples and trains in 15 epochs. The optimizer function is RMSprop at learning rate of 0.001 and the loss function is Contrasive Loss with margin of 0.5.
What did I miss and why doesn't my model learning?