3
$\begingroup$

In convolution layers, sometimes you need to pad some (usually 1 or 2 pixel) 0s at the edges of the original image before applying your convolution kernel (filter).

However, 0 has different meanings.

In the case of 0 - 255 gray scale images, 0 means black.

In the case of normalized images ( minus mean then divided by std), 0 is the mean value.

Should I always pad 0, or should I sometimes pad mean value ?

If in some cases I should pad mean value, what are those cases?

Edit:

And when should I use reflective padding?

Further more, is reflective padding always better than zero padding?

$\endgroup$
2
  • $\begingroup$ Why do you pad the pixels? Why with zero? $\endgroup$ – user2974951 Sep 25 '18 at 9:14
  • $\begingroup$ @user2974951 1.Because I have conv layers in "same" mode. 2. Please read my post, "why with zero" is part of my question. I saw many people use zero padding for its simplicity. However, I want to figure out why and when we should use zero padding. $\endgroup$ – user10253771 Sep 25 '18 at 9:47
3
$\begingroup$

Mean, zero, duplication (i.e., constant extrapolation), reflection, and symmetric padding are all valid and widely used methods of padding for conv layers. To my knowledge, there is no systemic study that says one is better than the other in all cases. In other words, one is not always better than any other. (Ideally, one would vary the padding type as a hyper-parameter I suppose.) I think intuitively that reflection and symmetric padding alters the local image structure and global statistics the least. (Clearly, the zeros from zero padding on the boundary are not truly part of the image, and the network has to learn this). However, these have different problems with "realism": for instance, reflection padding will "duplicate" a chair leg on the boundary if the padding is large enough, which may present its own problems depending on the task.

Overall, I think it doesn't matter too much, but it depends on the task and setup. For instance, in generative modelling and image-to-image translation, reflection padding avoid some artifacts on the boundary, as noted in the CycleGAN paper, for example.

An interesting recent paper is Partial Convolution based Padding by Liu et al, where they sidestep this issue by essentially having the convolution completely ignore the boundary. This is a partial convolution in the sense that it ignores the part of the filter that reaches outside of the image. The overall improvements seem to be relatively marginal, except that it does seem to help significantly on the boundary, as one would expect. See also this earlier work doing a similar thing in the segmentation context and this work which applies partial convolutions to inpainting. This approach to avoiding padding is seemingly well-principled.


TL;DR: test different paddings and see which is best for your architecture/task (also check the literature). If that's too much work, I default to reflect padding. If you want to get fancy or worry about boundary effects, use partial convolutions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.