3
$\begingroup$

We all know the CTRL+F "Find text..." feature in text editors / browsers.
I'd like to study the available algorithms to do something similar on an image.

Example of UI/UX: let's say you have an electronic schematic. You draw a rectangle around a diode, and then the algorithm will automatically find all similar diodes on this image.

In order to find a pattern on an image, I know (and have already used some of them) the classical tools:

  • openCV matchTemplate (pros: works with a single training example, cons: doesn't support rotation or scaling)
  • YOLO (pros: I thing it accepts rotation and scaling, cons: it requires 100s or 1000s of training examples)

Which available algorithms are there which would do their best with 1 or 2 training examples only and accept scaling / rotation?

$\endgroup$
4
  • 1
    $\begingroup$ You could try using a siamese network for this (cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf). The idea is not to classify images but to train a neural network to encode similar images in a similar way. You can train it once on a dataset (generating some similar and sissimilar pairs) and use it on new patterns afterwards. $\endgroup$
    – mprouveur
    Nov 23, 2021 at 10:29
  • $\begingroup$ @mprouveur can it work on segmentation of an image too? example: input=big electronic schmetatic, output=all the diodes are selected with a rectangle, i.e. they are localized on the image. From what I read in the PDF, it seems to (only?) classify pairs of images as similar or dissimilar. Can it also do localization of patterns in a bigger image? Do you have examples about this? Thanks $\endgroup$
    – Basj
    Nov 26, 2021 at 9:46
  • 1
    $\begingroup$ You can check SIFT, SURF. These are some of the scale and rotation invariant feature extraction algorithms. These algorithms can be used for finding the matching between source and target image $\endgroup$ Nov 27, 2021 at 5:07
  • $\begingroup$ you can use this as a patch classifier and use it to detect object instances inside an image. here's a blog post on pig detection on images : erdem.pl/2021/02/… In this example it is only used to detect one class (pig) but it can be used for any number of classes. (I agree with @UdayaUnnikrishnan SIFT should be a faster first approach) $\endgroup$
    – mprouveur
    Nov 29, 2021 at 15:17

1 Answer 1

2
+50
$\begingroup$

Invariant object recognition(IOR), refers to rapid and accurate recognition of objects in the presence of variations such as size, rotation and position.

SIFT and SURF are the most popular among them, but unfortunately both SIFT & SURF are patented.

If you are looking for opensource algorithm, I would suggest to go for openCV's Oriented FAST and rotated BRIEF (ORB), OpenCV ORB reference link
quotes

ORB is a good alternative to SIFT and SURF in computation cost, matching performance and mainly the patents.

RESEARCH papers

First two research papers, I found that dealt with one training example.

  • Rotation Invariant Object Recognition from One Training Example

    This paper presented a rotation invariant local descriptor based on the Gaussian derivatives. We use “steerable filter” to implement the derivative responses. Rotation invariance is achieved by “steering” the descriptor to the main orientation at a center pixel location. An advantage of this strategy is that the main orientation can be computed directly from the first order derivative responses. They also consider feature selection in the case where only a single example is available. Virtual images are generated by rotating and rescaling the image. Rotationally and scale unstable features are computed by estimating the main orientation at the center pixel location and count the number of correctly estimated. Unstable features are removed during the learning step. The resulting object recognition system performs robustly under various illumination changes, viewpoint changes, scale changes, and rotation in the image plane, and under partial occlusion

    Go for this approach if you want to start with one training example and wish to add more examples when they become available like an on-line learning scheme

  • Object Recognition from Local Scale-Invariant Features introduces SIFT

    This paper presents a new method for image feature generation called the Scale Invariant Feature Transform (SIFT). This approach transforms an image into a large collection of local feature vectors, called SIFT keys. Each image generates order of 1000 SIFT keys. These keys are used as input to a nearest-neighbor indexing method that identifies candidate object matches This system can learn an object model from a single image.

  • This research paper ORB: an efficient alternative to SIFT or SURF compares performance of ORB with SIFT & SURF with two images set, an indoor and an outdoor scene. ORB outperformed SIFT & SURF images taken in outdoor and almost same performance for indoor one

Below papers supports object recognition with few training examples only(not with rotation/scaling)

  • Few-Example Object Detection with Model Communication

    This paper considers the problem of generic object detection with very few training examples (bounding boxes) per class, named “few-example object detection (FEOD)”. Existing works on supervised/semi-supervised/weaklysupervised object detection. Multi-modal Self-Paced Learning for Detection (MSPLD) algorithm was proposed which combines self-paced learning and the multi-modal learning
    Unfortunately there is no existing implementation in python package as mentioned here. This paper though focus on few training samples for object detection, however doesn't support rotation and scaling of the objects

  • Object Recognition from very few Training Examples for Enhancing Bicycle Maps

    This paper introduces a system for object recognition that is trained with only 15 examples per class on average. To achieve this, we combine the advantages of convolutional neural networks and random forests to learn a patch-wise classifier. In the next step, we map the random forest to a neural network and transform the classifier to a fully convolutional network. Thereby, the processing of full images is significantly accelerated and bounding boxes can be predicted. However, this paper doesn't support rotation and scaling of the objects

POPULAR ALGORITHMs

  • Scale Invariant Feature Transform (SIFT)

  • Speeded Up Robust Feature (SURF): as SIFT was slow in performance a speeded up version SURF was introduced

  • Features from Accelerated Segment Test (FAST) corner detector: Feature detection methods weren't fast enough in SIFT & SURF

  • BRIEF (Binary Robust Independent Elementary Features)

    SIFT uses a feature descriptor with 128 floating point numbers. Consider thousands of such features. It takes lots of memory and more time for matching. We can compress it to make it faster. But still we have to calculate it first. There comes BRIEF which gives the shortcut to find binary descriptors with less memory, faster matching, still higher recognition rate.

  • Oriented FAST and rotated BRIEF (ORB)

    OpenCV devs came up with a new “FREE” alternative to SIFT & SURF, and that is ORB. good alternative to SIFT and SURF in computation cost, matching performance and mainly the patents

  • RIFT: is a rotation-invariant generalization of SIFT

  • RootSIFT

  • G-RIF

  • PCA-SIFT

  • Gauss-SIFT

  • KAZE and A-KAZE (KAZE Features and Accelerated-Kaze Features)

  • GLOH (Gradient Location and Orientation Histogram)

Below algorithm supports object recognition with few training examples only(not with rotation/scaling)

  • Fast R-CNN
  • R-FCN

    Object Detection via Region-based Fully Convolutional Networks. paper and package. Unfortunately does not support CPU-only mode

  • Detectron2

    Detectron2 is Facebook AI Research's next generation library that provides state-of-the-art detection and segmentation algorithms. It is the successor of maskrcnn-benchmark and Detectron. Detectron implements these object detection algorithms: Mask R-CNN ,RetinaNet, Faster R-CNN, RPN, Fast R-CNN, R-FCN

The following algorithms also outperforms SIFT and SURF

  • KAZE and A-KAZE
  • FAST corner detector
  • ORB
$\endgroup$
8
  • 2
    $\begingroup$ Part of this answer appear to have been copied from external sources, without indication that they are not your words. Specifically, "ORB is a good alternative ..." seems copied from opencv24-python-tutorials.readthedocs.io/en/latest/py_tutorials/…. Please revise this answer to properly credit the original source of all copied material: datascience.stackexchange.com/help/referencing $\endgroup$
    – D.W.
    Dec 27, 2021 at 6:14
  • 2
    $\begingroup$ Since this appear to have affected a number of your answers across the network, may I ask you to go through each one of your previous answers to check for similar instances of failure to credit external sources and edit all of them so they are brought into compliance with our rules? Thank you. $\endgroup$
    – D.W.
    Dec 27, 2021 at 6:15
  • 2
    $\begingroup$ I explained the problem several times on multiple of your posts. Perhaps haven't read those comments on the other posts yet? The problem is that your answer does not comply with our rules. I have given you a link where you can read about the rules. In particular, copying from an external source and then including somewhere in your answer a link to that external source does not comply with our rules and is not sufficient. $\endgroup$
    – D.W.
    Dec 28, 2021 at 19:20
  • 2
    $\begingroup$ I encourage you to read the rules to see what is sufficient. For example, you must indicate clearly which words were copied, using a Markdown quotation block (or possibly using quotation marks "..."). But don't take my words as the authoritative explanation of what the rules are -- please carefully read the link at datascience.stackexchange.com/help/referencing, and then review your answers and revise them to comply with it. $\endgroup$
    – D.W.
    Dec 28, 2021 at 19:21
  • 2
    $\begingroup$ I notice that I tried to explained the problem in my comment here ("without indication that they are not your words"), and in a comment elsewhere ("See datascience.stackexchange.com/help/referencing for our rules on this. Just including a link to the article somewhere is not sufficient -- see that link for the specific rules."), and in other comments that I can no longer view because the answer has since been deleted. Thank you for your attention to this. $\endgroup$
    – D.W.
    Dec 28, 2021 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.