I have a large image collection and wish to identify the images within that collection that appear to copy other images from the collection.

To give you a sense of the kinds of image pairs that I wish to classify as matches, please consider these examples:

enter image description here

I have hand classified roughly .25M pairs of matching images, and now wish to use those hand labelled matches to train a neural network model. I am just not sure which architecture would be ideally suited for this task.

I originally thought a Siamese Network might be appropriate, as they have been used for similar tasks, but the output from those classifiers seems more ideally suited to finding different figurations of the same object (which is not what I want), rather than different printings of the same figuration (which is what I want).

If anyone can help recommend papers or architectures ideally suited to identifying images given the training data I have prepared, I would be tremendously grateful for any insights you can offer.

  • $\begingroup$ "but the output from [Siamese networks] classifiers seems more ideally suited to finding different figurations of the same object" - I actually think Siamese networks would work perfectly. They learn what you tell them to learn. Others have made them learn "different figurations", but you could make them learn image duplication. $\endgroup$ – kbrose Dec 18 '18 at 14:37
  • $\begingroup$ I've just discovered a paper on DEep Local Features (DELF) that combines keypoints based analysis with convolutional neural networks to capture image similarity. Google has created a sample Colab notebook that implements DELF. $\endgroup$ – duhaime Jan 31 '19 at 18:29
  • $\begingroup$ Just a thought, what about variational autoencoders and measure the reconstruction error? I think it would work very well for the left two, but I'm not sure about the right examples. For those, I think a localized approach is required. $\endgroup$ – ldmtwo Sep 19 '19 at 19:26

U need to read about triplet loss function. Triplet loss function gets result embeddings from a network, that process 3 images by a network (2 similar and 1 non-similar) for one step: enter image description here

After that loss computing as: enter image description here

For more details read the paper from triplet loss authors.

Also may help PSNR, but this is not Deep Learning.

  • $\begingroup$ The triplet loss is a good suggestion, but link only answers are a bad fit for stack exchange. Can you explain more what the triplet loss is and why it would accomplish what OP wants? $\endgroup$ – kbrose Dec 18 '18 at 14:03
  • $\begingroup$ "Links to external resources are encouraged, but please add context around the link so your fellow users will have some idea what it is and why it’s there. Always quote the most relevant part of an important link, in case the target site is unreachable or goes permanently offline." - datascience.stackexchange.com/help/how-to-answer $\endgroup$ – kbrose Dec 18 '18 at 14:15
  • $\begingroup$ @kbrose I can do copy-paste from a link. Do u think I'm need to do that? Or u ask for TL;DR? $\endgroup$ – toodef Dec 18 '18 at 14:15
  • $\begingroup$ Removed my downvote. Thank you for adding some context @toodef. $\endgroup$ – kbrose Dec 18 '18 at 14:35
  • 1
    $\begingroup$ Sure, I appreciate that point of view. I guess I’m more of the opinion that if you don’t have time then just make a comment. $\endgroup$ – kbrose Dec 18 '18 at 15:19

If the images are more similar like you posted, you can go with Structural similarity index which gives output in the range -1 to 1. any thing more than 0.9 can be considered similar.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.