If we have a look to 90-99% of the papers published using a CNN (ConvNet). The vast majority of them use filter size of odd numbers:{1, 3, 5, 7} for the most used.

This situation can lead to some problem: With these filter sizes, usually the convolution operation is not perfect with a padding of 2 (common padding) and some edges of the input_field get lost in the process...

Question1: Why using only odd_numbers for convolutions filter sizes ?

Question2: Is it actually a problem to omit a small part of the input_field during the convolution ? Why so/not ?

up vote 7 down vote accepted

The convolution operation, simply put, is combination of element-wise product of two matrices. So long as these two matrices agree in dimensions, there shouldn't be a problem, and so I can understand the motivation behind your query.

A.1. However, the intent of convolution is to encode source data matrix (entire image) in terms of a filter or kernel. More specifically, we are trying to encode the pixels in the neighborhood of anchor/source pixels. Have a look at the figure below: enter image description here Typically, we consider every pixel of the source image as anchor/source pixel, but we are not constrained to do this. In fact, it is not uncommon to include a stride, where in we anchor/source pixels are separated by a specific number of pixels.

Okay, so what is the source pixel? It is the anchor point at which the kernel is centered and we are encoding all the neighboring pixels, including the anchor/source pixel. Since, the kernel is symmetrically shaped (not symmetric in kernel values), there are equal number (n) of pixel on all sides (4- connectivity) of the anchor pixel. Therefore, whatever this number of pixels maybe, the length of each side of our symmetrically shaped kernel is 2*n+1 (each side of the anchor + the anchor pixel), and therefore filter/kernels are always odd sized.

What if we decided to break with 'tradition' and used asymmetric kernels? You'd suffer aliasing errors, and so we don't do it. We consider the pixel to be the smallest entity, i.e. there is no sub-pixel concept here.

A.2 The boundary problem is dealt with using different approaches: some ignore it, some zero pad it, some mirror reflect it. If you are not going to compute an inverse operation, i.e. deconvolution, and are not interested in perfect reconstruction of original image, then you don't care about either loss of information or injection of noise due to the boundary problem. Typically, the pooling operation (average pooling or max pooling) will remove your boundary artifacts anyway. So, feel free to ignore part of your 'input field', your pooling operation will do so for you.

--

Zen of convolution:

In the old-school signal processing domain, when an input signal was convolved or passed through a filter, there was no way of judging a-prior which components of the convolved/filtered response were relevant/informative and which were not. Consequently, the aim was to preserve signal components (all of it) in these transformations.

These signal components are information. Some components are more informative than others. The only reason for this is that we are interested in extracting higher-level information; Information pertinent towards some semantic classes. Accordingly, those signal components that do not provide the information we are specifically interested in can be pruned out. Therefore, unlike old-school dogmas about convolution/filtering, we are free to pool/prune the convolution response as we feel like. The way we feel like doing so is to rigorously remove all data components that are not contributing towards improving our statistical model.

  • Thank you for your exaplantion, but isnt it easier to understand this by looking at the definition of convolution in mathematical terms? en.wikipedia.org/wiki/Convolution#Discrete_convolution Because here we see that we will always have m terms to left and right of the origin... hence this will always result in an uneven amount. – zwep Apr 24 at 13:16
  • @zwep Agreed! Q&A forums do not replace encyclopedias; but merely supplement them. However, a broader perspective to your comment: Definition is typically merely peer reviewed best choice among available options; it isn't gospel truth. Nevertheless, rules must first be learnt before breaking them, so it can be clearly communicated to the community exactly how and why you are doing it differently. The reasons and alternate choices behind a certain definition are in research papers and typically not in an encyclopedia. – Dynamic Stardust Apr 28 at 16:07

1) Suppose input_field is all zero except for one entry at index idx. An odd filter size will return data with a peak centered around idx, an even filter size won't - consider the case of a uniform filter with size 2. Most people want to preserve the locations of peaks when they filter.

2) All of the input_field is relevant for the convolution, but the edges of output_field cannot be computed accurately since the necessary data is not contained in input_field. If I want to compute an answer for the first element of output_field, the filter must be centered on the first element of input_field. But then there are filter elements that don't correspond to any available element of input_field. There are various tricks to get a guess for the edges of output_field.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.