I'm trying to create a facial recognition detector using triplet loss followed by a kNN algorithm.

I have roughly 10000 input images with 3 different classes, input size is 80x80. Model structure uses resnet with imagenet weightings, followed by a few dense layers for embeddings:

base_cnn = resnet.ResNet50(
    weights="imagenet", input_shape=image_input_shape, include_top=False
trainable = False
for layer in base_cnn.layers:
    layer.trainable = trainable
x = Flatten()(base_cnn.output)
x = Dense(128, activation="relu", kernel_regularizer=tf.keras.regularizers.l2(0.0001))(x)
x = BatchNormalization()(x)
x = Dropout(0.5)(x)
output = Dense(embedding_size)(x)
embedding = Model(base_cnn.input, output, name="Embedding")

The problem is that the training loss is small, but the validation loss is barely decreasing (if at all): Loss over 50 epochs

I assume this is due to overfitting, however I've added in everything I can think of to prevent this (dropout, regularisation and I've played with the learning rate). I'm guessing that the dataset size is the problem, but hoped by using transfer learning I'd be able to get decent results with this dataset.

Any suggestions on how to interpret this, and how to improve validation loss?

Thanks in advance.

  • $\begingroup$ How did you choose triplets? $\endgroup$
    – B200011011
    Feb 16 at 17:43

1 Answer 1


I probably don't understand the whole context but I see there could be a couple problems:

  1. You want to use pretrained model by classification on ImageNet for face recognition task. Those are very different task and the representation from pretrained ResNet will not be very useful for face recognition.
  2. You have just 3 classes. Do you mean just 3 person? That is not enough for learning general representation extractor. How do you even split it between train and val?

I see you have a couple options:

  • Don't train your face recognition system but use of the shelf like this one.
  • Train some layers even from base_cnn
  • Download some face recognition dataset (like Labeled Faces in the Wild or preferably larger), pretrain model on it and finetune (or just validate) on your dataset.
  • $\begingroup$ The idea was that I have a feature extractor layer then an embedding layer, ImageNet was being used to create the features. I have 3 people at the moment, but I can add more in, I assumed that this wouldnt be a problem - how many people would be required to allow it to extract the general representation? I split the data from train and val as I have labels corresponding to an image stored in a dataframe, which is then split to a ratio of 70, 20, 10 for train val and test. The primary issue is Im using infrared images, not visible, so using pretrained classifiers and open data wont work. $\endgroup$
    – Jkind9
    Sep 8, 2021 at 8:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.