2
$\begingroup$

There is quite a good explanation which fully comply with my vision. But seems it lacks one final step. As Jean states, moving an object significantly in the input image will cause the change in which neuron is activated in the yellow layer (the one previous to the first fully connected layer). So that we see that the part of the network before FCL is position equivariant. Then author says that because the network detects an object at any location, the FCLs should have taken care of it.

(Tried this demo but it doesn't seem to address the second question)

$\endgroup$
1
$\begingroup$

In your link, the author Jean states:

Additionally, I believe that if a CNN is trained showing faces only at one corner, during the learning process, the fully-connected layer may become insensitive to faces in other corners.

I also believe this to be correct. The FCN does not in any way add or improve translation invariance. Instead it will treat all outputs - each individual pixel of the "feature maps" - of the last convolutional layer as entirely different features. It must be trained with enough examples in order to generalise well.

However, the feature maps are not themselves simple, clean detectors of objects as the simplified explanation of CNNs might imply. In a deep network they can be very complex and respond to a wide range of stimuli. They will also respond somewhat fuzzily, so that e.g. an eye or the side of a head showing an ear can trigger multiple feature map pixels (to the feature map's kernels, the same object slightly translated will look like distorted version of the same feature, and will still match enough sub-components of the object to trigger a positive response). The last layer will not necessarily detect full objects, but significant chunks of objects, areas of important texture etc. So position can still be quite fluid, and the "head detector one pixel off in last convolutional layer" scenario is not particularly realistic - although it may affect relative strength/confidence of predictions.

This can still be a problem if you need your network to generalise. If you suspect that your training data might not be covering enough variations in position, orientation etc of images, then a common approach is data augmentation. As a reflected, rotated or cropped image of an object should usually be classified as the same object, then you can pre-process your training data using those transforms to make many random variations of the input images. Some deep learning frameworks will allow you to do this continuously, generating fresh images for each and every batch.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, that's great explanation of the problem. Could you also comment regarding what did you mean by "CNNs are translation equivariant, and that is a key feature explaining the efficiency". Did you mean "if trained with sufficient set of samples" (but it looks they are invariant in that case)? $\endgroup$ – VladimirLenin Jul 14 '17 at 15:53
  • $\begingroup$ @VladimirLenin: Equivariance is due to re-using the kernels in each position. It is efficient because detecting edges, corners, circles and at higher levels textures, shapes, is the same process and equally useful thing to do at all locations in an image. That means the kernels, which have only a few weights each, get to be re-used a lot. This is not always true for all data types, but in photographic images, audio and similar "same signal type repeated across a structure" it seems very useful. $\endgroup$ – Neil Slater Jul 14 '17 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.