I am working on a face recognition application using deep learning. To plot the ROC curves and do performance evaluation, I extracted the features from the last layer of the deep neural network and I have tried Euclidian and Manhattan distances between high dimensional feature representation, but this won't give me accurate measure in terms of comparison as faces have intra-personal and interpersonal differences and other challenges such as pose, lighting condition, and expression. I came across techniques such as joint bayesian, sparse projection, KL-divergence based regularizer/prior(EM algorithm) etc. Are there any other approaches that I'm missing here? What would be the efficient approach to generate similarity scores?

  • 1
    $\begingroup$ It depends on how (if?) you learned the representation. If you do it right you can use any reasonable function you want, including the cosine similarity. The ones you mentioned are distances or metrics, by the way; not similarities. So if the representation is fixed (e.g., because you're using a pretrained network) you can learn a similarity or metric on top: en.wikipedia.org/wiki/Similarity_learning $\endgroup$
    – Emre
    Nov 26, 2017 at 0:01

1 Answer 1


A common method is defining "landmarks", a set of specific points that exist on every face . For example, the outside edge of each eye or the bottom of the chin are landmarks on most faces. Train the deep learning network to learn the relationship between these landmarks which will form the embedding space. Then define a distance metric in this embedding space. Finally, create a threshold for similarity between sets of points in this embedding space. That threshold can be learned through "one-class" classification techniques.

One gotcha is pose. Faces turned different directions look totally different to a computer. Assuming you have isolated the faces in an image, it is best practice to warp each face so that the eyes and lips are always in the place in the image. This (and other normalization steps) should be done early in the processing pipeline to make subsequent analysis much easier.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.