5
$\begingroup$

Suppose a convolution neural network is trained on small images of an object, say flower, as in following 3 training images:

enter image description here

Will this CNN correctly classify if the same object is present in zoomed form in a test image? As in following example:

enter image description here

What if the situation is reverse, i.e. training on large sized objects and testing with small sized object?

What is the best way to take care of different size of object that may be present for correct classification of images?

(The images are from: https://www.shutterstock.com/video/search/flowers )

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

I think more information on the project can help. For instance in your example, you just need to detect the flower frame in the whole image and crop it. Then you simply resize your images to be the same size before feeding your NN. In this case, detecting the target frame and scaling it is a part of pre-processing and the learning phase stays the same.

Another approach is to learn this scaling. It means that the learning algorithm needs scale-invariant features but NNs are supposed to actually create features in their hidden layers through learning. So if you have sufficiently big dataset in which all variants are presented enough, your NN will learn it (most probably with poor results). The process is intuitive ML-wise:

  • NN extract features from input images i.e. when the label of same object with different sizes are the same, it captures those features which discriminate these objects in a scale-invariant manner.
  • This was simplified! The trick is that CNN can learn resizing using a technique called Pooling. But you are the one who plays with parameters to make your NN working!

Probably the best is the situation in which your dataset is not that huge and rich or you do not want to play with many parameters or you want to increase the quality of results. Then focus on finding scale-invariant features. Either you extract them from your images and feed it to the NN, or you design NN in a way that it finds them itself. A more intuitive way is to train different models on different scales and ensemble them.

Hope it helps :) Good Luck!

PS: SIFT is patented. Be careful not to use it for commercial purposes.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for a good answer. So a CNN with MaxPooling layers is scale-invariant. Am I right? $\endgroup$
    – rnso
    Commented Oct 18, 2018 at 1:23
  • $\begingroup$ To some extent yes. The point is that, to the best of my knowledge, parameter tuning for this porpuse is not that trivial. Where would you place your pooling layer? How do you know about the scale variety in dataset to determine the window size in pooling? answering these questions needs lots of parameter tuning. $\endgroup$
    – Kasra Manshaei
    Commented Oct 18, 2018 at 6:50
  • $\begingroup$ The white part of training images are not really just white or blank. They contain random colors/designs/objects. But presence of small flowers in any image indicate a flower label for that image. Does this make a difference to your recommendations? $\endgroup$
    – rnso
    Commented Oct 18, 2018 at 7:16
  • $\begingroup$ Then it goes more toward the first suggestion. First detect object in each image and crop it then start anything else. Viola-Jones algorithm can detect your objects. Then you can go through any of suggested ways. $\endgroup$
    – Kasra Manshaei
    Commented Oct 18, 2018 at 10:49
2
$\begingroup$

In general, No, CNNs are not scale-invariant. The proof for this is simple: The maximum receptive field for a convolutional network is always fixed and finite, and it is a function of the network's depth, the number and rate of downsampling layers, and of the kernel sizes and strides. All one has to do is give an input that is scaled to exceed the network's maximum receptive field, and you will arrive at a point at which it will not be able to identify the object robustly.

Try tinkering with the Receptive Field Calculator here to see some concrete and interactive examples. Since the receptive field is always bounded, the ability of a CNN to match objects is limited to a pixel-space scale comparable to the maximum theoretical receptive field size.

Beside the receptive field matter, the behavior of a network that is trained at one scale and tested at another is generally not well-defined. Don't expect it to work. Many papers on recognition tasks improve performance by making the scale of inputs more attuned to the scale expected by the network. This is a key to the success of Spatial Transformer Networks.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.