Difference between Siamese Network and Prototypical Networks for One Shot Learning

Question

I am having a bit of trouble understanding how the architecture of prototypical networks in a one shot learning use case differs from Siamese networks.

If I’m understanding correctly, Siamese networks maps pairwise examples using same weights and biases into latent space and computes a similarity function. Prototypical networks finds the least distances between query and the mean of the examples in latent space (but in a one shot learning case, the mean would just be the latent vector itself?). I’m confused as to how prototypical networks would compare each example if they are in different latent spaces, so I’m assuming they all are mapped using the same neural network, with same weights and biases as well.

Answer 1

For unseen classes at inference time, one-shot learning is quite similar when using Siamese networks and prototypical networks. Both approaches involve using your trained network to generate

1. an embedding for the example you wish to classify (the query),
2. an embedding for your one known example of each class

Then, you compute distances between 1) and each embedding from 2) to determine the best class.

However, the two approaches differ in how the network is trained. Siamese networks are trained to produce embeddings that are nearby for examples from the same class and far away for examples from different classes. In other words, the network gets pairs of examples during training. Pairs are either from the same class or from randomly selected other classes, so the model ultimately learns an embedding space that embeds examples from all classes seen during training into different regions -- but it learns to do this based on a loss function that evaluates the distance between pairs of embeddings.

Prototypical networks, on the other hand, are trained to produce embeddings that lead to successful results on a k-way classification problem. From the original paper:

Training episodes are formed by randomly selecting a subset of classes from the training set, then choosing a subset of examples within each class to act as the support set and a subset of the remainder to serve as query points.

So during training, the model is not evaluating individual pairs selected from all classes; it's comparing query examples to a specific set of other classes and predicting the result, then doing the same thing again for a different set of query examples and a different set of classes, etc. The loss is based on classification performance.

The difference between these approaches is confusing because any model that does a good job on the Siamese network task ("make embeddings for members of the same class look similar") also does a good job on the prototypical network task ("make embeddings such that a member of a class is closer to the prototypical (mean) embedding of the other members of its class than it is to the prototypical embeddings of other classes"). But the training approach used to learn that embedding space is different.