1
$\begingroup$

I'm trying to train a Siamese Network model for a signatures dataset using Keras API and considering the loss only seems not bad. But ironically enough the model accuracy stuck at 0.5.

Model Loss:

Model Loss

Model Accuracy:

Model Accuracy

My model is kind of a deep model, here's its architecture:

input = Input((128, 128, 1))

x = BatchNormalization()(input)
x = Conv2D(16, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(32, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(64, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(128, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Conv2D(256, (2, 2), activation="tanh")(x)
x = AveragePooling2D(pool_size=(2, 2))(x)
x = Flatten()(x)

x = BatchNormalization()(x)
x = Dense(10, activation="tanh", kernel_regularizer="l2")(x)

embedding_network = Model(input, x)

input_1 = Input((128, 128, 1))
input_2 = Input((128, 128, 1))

tower_1 = embedding_network(input_1)
tower_2 = embedding_network(input_2)

merge_layer = Lambda(euclidean_distance)([tower_1, tower_2])
normal_layer = BatchNormalization()(merge_layer)
output_layer = Dense(1, activation="sigmoid")(normal_layer)

siamese = Model(inputs=[input_1, input_2], outputs=output_layer)

The model takes grayscale 128x128 images of signatures in batches of 16 images using the ImageDataGenerator class having 800 training samples and 200 validation samples and trains in 15 epochs. The optimizer function is RMSprop at learning rate of 0.001 and the loss function is Contrasive Loss with margin of 0.5.

What did I miss and why doesn't my model learning?

$\endgroup$
4
  • 1
    $\begingroup$ Is this binary classification? Is the dataset balanced? I don't know anything about DL and images, but I would start by looking at a confusion matrix to understand what happens: does the model always predict the same class across epochs? And if so why is the loss changing at all? It could be a bug somewhere. $\endgroup$
    – Erwan
    Commented Mar 31, 2022 at 23:38
  • $\begingroup$ How did you write the loss function? and how did you build your training triplets? $\endgroup$
    – Oscar
    Commented Apr 1, 2022 at 10:59
  • 1
    $\begingroup$ @Oscar its the contrasive loss function: square_pred = tf.math.square(y_pred) margin_square = tf.math.square(tf.math.maximum(margin - (y_pred), 0)) return tf.math.reduce_mean( (1 - y_true) * square_pred + (y_true) * margin_square ) $\endgroup$ Commented Apr 1, 2022 at 19:37
  • $\begingroup$ @Erwan its a siamese network it measures the similarity of two images and compares the predicted similarity with the defined similarity (0 non-similar or 1 similar) $\endgroup$ Commented Apr 1, 2022 at 19:43

1 Answer 1

2
$\begingroup$

Turns out the problem was hidden in the loss function. I decided to change it from Contrasive Loss to Categorical Crossentropy just for fun, and with some kind of "magic" it worked.

from:

def loss(margin=1):
  def contrastive_loss(y_true, y_pred):
    square_pred = tf.math.square(y_pred)
    margin_square = tf.math.square(tf.math.maximum(margin - (y_pred), 0))
    return tf.math.reduce_mean(
        (1 - y_true) * square_pred + (y_true) * margin_square
    )

  return contrastive_loss

siamese.compile(loss=loss(margin=MARGIN), optimizer=RMSprop(learning_rate=0.001), metrics=["accuracy"])

to:

siamese.compile(loss="categorical_crossentropy", optimizer=RMSprop(learning_rate=0.001), metrics=["accuracy"])
$\endgroup$
2
  • $\begingroup$ Curious to see how well this network generalizes. Are you testing signatures that are not found in the training dataset? You shouldn't change the loss function, because contrastive loss tries to separate different embeddings while bringing together similar ones. - That's what you want. You want distance between embeddings, independent of class. Categorical crossentropy is maximizing inter-class distances, which is what you want in a classification problem: samples belonging to a class get grouped together, but this assumes that you know where each class cluster lies in the manifold beforehand. $\endgroup$ Commented Jun 21 at 7:04
  • $\begingroup$ TL;DR: You shouldn't change the loss function. Contrastive Loss -> Learns to separate things, independent of their classes . Categorical Crossentropy -> Learns to group samples in their class, the number of classes need to be known at "train time". $\endgroup$ Commented Jun 21 at 7:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.