Siamese Neural Network for massive class variation

Question

I have case to check whether the person has been registered into a database,and if the images has high similiarity with one of the image in the db, i want to retrieve the image information (name, id, etc) to email or other party. The problem is the dataset are very limited (I only get 1 image per person) so i augmented the image and get about 30 variation of augmented faces. At first i have thinking using Siamese NN since it has good performance with limited data source, but realise its need pair of image (the gaithered and one from the dataset), loop through big dataset (need to loop through 3k of image), SNN is not too efficient. I afraid using VGG or Facenet since the augmented dataset are not too big enough to gain certain accuracy.

Is there any suggestion based on my case? The one idea i have to tackle it is to use NN as feature and use similarity calculation based from stored NN.

Answer 1

I have some discussion about this problem with my friend and found a good solution. The answer is to using CNN not as a classifier but as feature generator (embedded) and store it as vector. The next is to using vector similarities to check whether the person is same or not or who is in the picture.

Answer 2

You can use a pre-trained ResNet as you mentioned, and then fine tune it as part of an encoder in a Siamese network architecture. You don't need to update the weights after all combinations, you can use mini batches. This works quite well! Good luck!