Siamese vs matching network for correct image category matching

Question

I have to find the closest match between my image and bunch of already collected images of different classes in the folder. Whic meta-learning approach should I select. I am thinking about the Siamese or matching network. In Siamese, I have to match my image with all existing images in the folder to find the correct match. So do you think if I can use a matching network and produce a better result? What is the parameter based on which developer decides where to use matching/prototypical network and where to use Siamese network.

Note- At the time of the test new unseen images class can be added on which model is not trained. Do you think a matching network will still work for this case?

Answer 1

A standard approach for this kind of problem is to take the representations of the images computed by some pre-trained neural network (e.g. imagenet). As "representation" of the image we usually take the vector after flattening the output of the last convolutional layer block.

If your images are not of general domain, but are of a very specific domain, and you have enough data, you may want to train an autoencoder on them instead of using a pre-trained model.

You now compute and store the representations of all the images you want to compare against. Once you have the representations, for each image you want to match, you simply compute the closes image in the representation space. There are libraries like Facebook's faiss that are very fast computing binary similarities over a large number of vectors.

This kind of approach is cheap in terms of compute, it is scalable and it is widely used.