4
$\begingroup$

I have a database with around 30,000 pictures. All of them are a different object. They are all from a certain perspective, the pictures itself are the same size but the objects vary in size. I want to build a system that you can query with a new picture, and that will return it's nearest neighbor, given that it is similar enough. The queried images will look relatively similar to the originals, there might be some horizontal and or vertical translations, a bit different lightning and sometimes there will be a sticker on a different place. Some queried objects will not be in the set, and that needs to be returned as well. What are good techniques to try and what would be the downside? Getting multiple pictures of each object is infeasible. Here's some ideas, I'm wondering if there is more to try:

  • Euclidean distance on the raw data (very sensitive but fast)

  • Use traditional keypoint matching, linear matching is very slow unfortunately

  • Use (denoising) autoencoder for lower dimensional feature representation, linear match on this encoded space (smallest Euclidean distance, at least faster linear search)

  • Learn siamese network for linear matching (don't know how fast this works but seems slow too)

  • Learn deep binary autoencoder onto 28 bits which allows for very quick narrowing of the search space to do one of the previous methods, by using these bits as memory mapping to a list of candidate solutions

Any other ideas?

|improve this question
$\endgroup$
  • $\begingroup$ Convolving the images would give you a measure of similarity, and it would be fast. You could smooth the images first to limit noise effects too, and equalize the images. The weakness would be the translations though. $\endgroup$ – Hobbes Aug 22 '16 at 14:19
  • $\begingroup$ But what convolution to use? If that is the case isn't using a convolutional network to learn the best kernels to use to distinguish different images better? And then you would use Euclidean distance over the convolutions? Then we get into the (denoising) autoencoder territory correct? $\endgroup$ – Jan van der Vegt Aug 22 '16 at 14:54
  • $\begingroup$ What I'm referring to is actually the cross-correlation. Here's a reference: mathworks.com/help/images/examples/… $\endgroup$ – Hobbes Aug 22 '16 at 15:05
8
$\begingroup$

Hashing is the way to go if you want fast -- constant time -- retrieval of nearest neighbors. Here's a recent example using neural networks to learn a binary hash:

Deep Learning of Binary Hash Codes for Fast Image Retrieval (code) (slides)

You want to avoid doing all-pairs computations like correlations.

|improve this answer
$\endgroup$
  • $\begingroup$ They use this in a supervised manner, in my case there are no labels (although that might be a possibility for some of the photos). Do you think a similar approach could work with a convolutional autoencoder with the sigmoid in the middle, including the added binary 'enforcing' term in the loss function? $\endgroup$ – Jan van der Vegt Aug 23 '16 at 22:57
  • 2
    $\begingroup$ Here's the unsupervised version: Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks $\endgroup$ – Emre Aug 23 '16 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.