I've read about FaceNet but the main problem is still unclear for me. Does embedding work on the trained images only? Or once trained on a big dataset it will readily cluster unknown faces without re-training? I need an approach to add new faces automatically.


Yes embedding works on unseen images. That is the point of the approach - instead of learning to classify each face in the training set directly, the model learns a vector that is intended to be:

  • far apart when the face itself is different

  • similar when the face is the same, but other items in the image - e.g. background, hair style etc - are different

In order to use embeddings to recognise someone, you need one or more sample images of that person, so that you can generate a target embedding. Then you can predict any new image that generates a new embedding close enough to the target embedding(s) to be an image of the same person.

This can still suffer usual problems of overfitting to training set. So it is important to understand what training data was used for the model. Embeddings generated from images that are very different from the training data will not be as reliable as embeddings of images that are similar to the training data.

  • $\begingroup$ Then why they still use SVM on top of it sometimes? For me the most benefit of the approach is getting rid of SVM which requires re-training for new entities. $\endgroup$ Dec 28 '18 at 9:51
  • 1
    $\begingroup$ @VladimirLenin: That is a different question - perhaps ask it separately. In brief, SVM adds a classifier layer that works on the embedding vector. It is not necessary, but may increase accuracy in a "closed" set of faces. If you have an open set of faces (e.g. where you could have a large number of unknown identities), you have to avoid the use of a classifier and work directly with some distance metric between the embedding vectors. $\endgroup$ Dec 28 '18 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.