0
$\begingroup$

I want to determine the similarity between images based on different features. The images show the same type of object (e.g. cars). I want to order images based on their similarity (e.g. through a feature vector). There are ways to solve this, for example a convolutional neural network. However, the images may be taken from slightly different perspectives: Sometimes directly from the front, sometimes with a slight ankle. What I DONT want is images ordered by their perspective rather than by the actual object. My questions:

  • What can I do to avoid primary order by perspective
  • Do I have to expect the ankle of the picture to be a primary factor in sorting order or will it more likely be negligible?
  • Is there a way to extract/visualise the features the network uses (unsupervised) in order to manipulate the sorting order (e.g. by saying I want to increase the weight of THIS visual feature when ordering)?
$\endgroup$
0
$\begingroup$

Generative Adverserial Networks can be tolerant of this type of perturbation. In fact, if you look at figure 8 of the DCGAN paper, the authors demonstrate how the GAN learned to encode perspective directly such that the authors are able to use the GAN to rotate the camera on an image by applying a "turn vector".

$\endgroup$
1
  • $\begingroup$ This is a very interesting paper, thank you for suggesting it. Are you aware of any other techniques to solve this problem? I would appreciate getting an overview of how to approach these situations. $\endgroup$
    – Gegenwind
    Jan 16 '18 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.