Advantages of one shot learning over image classification

Question

This is a rather conceptual question. From what I've read I gather that one shot learning is useful for use cases in which you don't have datasets of millions of images of employees etc. By a one shot operation, the network or networks will do a similarity measure to determine whether the given image belongs to a bunch of images or not.

However, some of these siamese networks are trained on millions or hundreds of millions of images. So doesn't that defeat the purpose of one shot learning ? Why can't plain image classification + transfer learning be used instead ? For instance, in face recognition systems, for every image - classify, doesn't fall into any of the classes - reject. If this is a legitimate new employee image, add the image to the database and retrain the network using transfer learning.

Does one shot learning remove the need for this retraining ? Can someone help me with these basic intuitions please

Answer 1

In the case of face detection in particular it can get very use case specific:

1a) Let's say I am facebook, and I have a billion possible faces of users to tag. With the one-shot approach, I am storing a representation of each new face as you suggest, I assume it is something like One-shot Learning with Memory-Augmented Neural Networks. Now I have a problem with a 'memory' that there are billion faces, and that is a lot of computation to do for lookup. The representations learned must increase in size w.r.t to the dataset in order to be able to distinguish between instances, so I get hit again.

1b) With Deepface, which uses the siamese architecture you mention I have the advantage that to classify a new user, I just have to run it on maybe 1k images of friends around the network where that image comes from. I do not have to retrain the network if a new human is found, since after the initial training, there is not such a problem with doing a comparison between images, as the network is doing a one-vs-one, not a one-vs-all query. The network is not learning classes, but representations and the differences between representations that are most indicative of two examples being different classes.

2a) Now let's say I am a modeling agency with 800 employees, and I have 1000 examples per person. In that case, the one shot learning approach is feasible, especially if you have downloaded a pretrained network, and do the fine-tuning as you suggested. There is close to no cost for any projected growth (the one-shot learning paper trained on a 4million example dataset).

2b) In this case it may be feasible to finetune a network with just a final layer with 1000 logits (800 for employees, and a 200 for growth). This would be totally infeasible for the billion or even million classes (individual humans) - but still probably not advisable.

In short, it depends on how many examples per class you have and the number of classes you have. Many architectures out there only work when you have millions of images for relatively few classes (this applies to all the imagenet entries). Others are designed to learn from 10k examples or less, with 50 examples per class, and yet others are designed for not explicitly using the notion of classes, when even that becomes too big.

For a comprehensive review of face detection check out Deep Face Recognition: A Survey. New on arxiv 4/18/2018