5
$\begingroup$

As I understand STN as described by the the deepmind paper https://arxiv.org/abs/1506.02025

  • allow a neural network to learn how to perform spatial transformations on the input image in order to enhance the geometric invariance of the model.

whereas Deformable Convolutions https://arxiv.org/abs/1703.06211

  • add 2D offsets to the regular grid sampling locations in the standard convolution.

From the paper:

...deformable convolution does not adopt a global parametric transformation and feature warping. Instead, it samples the feature map in a local and dense manner. To generate new feature maps, it has a weighted summation step, which is absent in STN.

To me STNs augment the input image whereas DC augment the kernel shape. Results should be similar/same in terms of field of view of the network? Please correct me if mistaken.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

The main difference, as they mentioned in the paper, is that STN has a global parameter to transform the features. That is, it computes one set of parameters to transform the input. DC computes a 2D offset map for each location in the input, so for each location in the input features there is a separate transformation.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The 2D offset map is fixed for an input layer, so I am not sure what you mean by 'for each location in the input'. $\endgroup$
    – Benedict K.
    Commented Jun 12, 2018 at 16:23
  • $\begingroup$ The offset map is learned via a convolution of the input feature map. 'For each location in the input' I am referring to each pixel has its own 2D offset. This is illustrated in Figure 2 of the paper. $\endgroup$
    – kenny
    Commented Jun 12, 2018 at 20:11
  • $\begingroup$ Right but the offset is shared throughout the input feature map. So let's assume a 25x25x50 input map with h x w x nbr features then the offset is the same for each pixel x 50. There was some confusion about this on medium/reddit. $\endgroup$
    – Benedict K.
    Commented Jun 18, 2018 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.