In your link, the author Jean states:
Additionally, I believe that if a CNN is trained showing faces only at one corner, during the learning process, the fully-connected layer may become insensitive to faces in other corners.
I also believe this to be correct. The FCN does not in any way add or improve translation invariance. Instead it will treat all outputs - each individual pixel of the "feature maps" - of the last convolutional layer as entirely different features. It must be trained with enough examples in order to generalise well.
However, the feature maps are not themselves simple, clean detectors of objects as the simplified explanation of CNNs might imply. In a deep network they can be very complex and respond to a wide range of stimuli. They will also respond somewhat fuzzily, so that e.g. an eye or the side of a head showing an ear can trigger multiple feature map pixels (to the feature map's kernels, the same object slightly translated will look like distorted version of the same feature, and will still match enough sub-components of the object to trigger a positive response). The last layer will not necessarily detect full objects, but significant chunks of objects, areas of important texture etc. So position can still be quite fluid, and the "head detector one pixel off in last convolutional layer" scenario is not particularly realistic - although it may affect relative strength/confidence of predictions.
This can still be a problem if you need your network to generalise. If you suspect that your training data might not be covering enough variations in position, orientation etc of images, then a common approach is data augmentation. As a reflected, rotated or cropped image of an object should usually be classified as the same object, then you can pre-process your training data using those transforms to make many random variations of the input images. Some deep learning frameworks will allow you to do this continuously, generating fresh images for each and every batch.
