7
$\begingroup$

Right now I'm studying Convolutional Neural Networks.

Why must a CNN have a fixed input size?

I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and I also know that it is due to the fully connected layers placed at the end of the network.

But why? I can not understand what the presence of the fully connected layers implies and why we are forced to have a fixed input size.

$\endgroup$
  • 2
    $\begingroup$ You can also overcome this with a global pooling layer between the convolutional and dense layers. $\endgroup$ – Ben Reiniger Nov 30 '19 at 17:19
8
$\begingroup$

I think the answer to this question is weight sharing in convolutional layers, which you don't have in fully-connected ones. In convolutional layers you only train the kernel, which is then convolved with the input of that layer. If you make the input larger, you still would use the same kernel, only the size of the output would also increase accordingly. The same is true for pooling layers.

So, for convolutional layers the number of trainable weights is (mostly) independent of input and output size, but output size is determined by input size and vice versa.

In fully-connected layers you train weight to connect every dimension of the input with every dimension of the output, so if you made the input larger, you would require more weights. But you cannot just make up new weights, they would need to be trained.

So, for fully-connected layers the weight matrix determines both input and output size.

Since CNN often have one or more fully-connected layers in the end, there is a constraint on what the input dimension to the fully-connected layers has to be, which in turn determines the input size of the highest convolutional layer, which in turn determines the input size of the second highest convolutional layer and so on and so on, until you reach the input layer.

| improve this answer | |
$\endgroup$
  • $\begingroup$ This is true only if we're talking about "same" padding right? Even with fully conv networks, if i don't use "same" padding, i could reach a point where my filter is bigger than my image (and makes no sense for convolution) right? $\endgroup$ – Mattia Surricchio Nov 30 '19 at 17:06
  • 1
    $\begingroup$ I see what you mean. That's why I wrote that weights are "(mostly)" independent of the input size. If you don't use valid padding or do pooling or use strides > 1, you shrink the image, which does not change anything about the argument, until you run into the edge case, where there is only a single pixel left. But then you could still convolve using proper padding. I'm not sure how reasonable such a shrinkage would be and I assume you get some weird behaviour. This kind of imposes a minimal input size for a given network architecture, but this would also be true for fully-conv networks. $\endgroup$ – matthiaw91 Nov 30 '19 at 18:45
6
$\begingroup$

It's actually not true. CNN's don't have to have a fixed-size input. It is possible to build CNN architectures that can handle variable-length inputs. Most standard CNNs are designed for a fixed-size input, because they contain elements of their architecture that don't generalize well to other sizes, but this is not inherent.

For example, standard CNN architectures often use many convolutional layers followed by a few fully connected layers. The fully connected layer requires a fixed-length input; if you trained a fully connected layer on inputs of size 100, and then there's no obvious way to handle an input of size 200, because you only have weights for 100 inputs and it's not clear what weights to use for 200 inputs.

That said, the convolutional layers themselves can be used on variable-length inputs. A convolutional layer has a convolutional kernel of fixed size (say, 3x3) that is applied to the entire input image. The training process learns this kernel; the weights you learn determine the kernel. Once you've learned the kernel, it can be used on an image of any size. So the convolutional layers can adapt to arbitrary-sized inputs. It's when you follow a convolutional layer with a fully connected layer that you get into trouble with variable-size inputs.

You might be wondering, if we used a fully convolutional network (i.e., only convolutional layers and nothing else), could we then handle variable-length inputs? Unfortunately, it's not quite that easy. We typically need to produce a fixed-length output (e.g., one output per class). So, we will need some layer somewhere that maps a variable-length input to a fixed-length output.

Fortunately, there are methods in the literature for doing that. Thus, it is possible to build networks that can handle variable-length inputs. For instance, you can train and test on images of multiple sizes; or train on images of one size and test on images of another size. For more information on those architectures, see e.g.:

and so on.

That said, these methods are not yet as widely used as they could be. Many common neural network architectures don't use these methods, perhaps because it is easier to resize images to a fixed size and not worry about this, or perhaps because of historical inertia.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Input size determines the overall number of parameters of the Neural Network. During training, each parameter of the model specializes to "learn" some part of the signal. This implies that once you change the number of parameters, the whole model must be retrained. That's why we can't afford to let the input shape change.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Why isn't this valid also for fully convolutional neural networks? I mean, from what i've learnt, they are capable of managing input with many sizes $\endgroup$ – Mattia Surricchio Nov 30 '19 at 17:07
  • 1
    $\begingroup$ Inputs of various sizes are managed in different ways a priori. For example, images are scaled to the same size before training; alternatively, images smaller that the input size get zero padded. But in any case, the size of the input matrix is fixed. Let me know if this answers your question. $\endgroup$ – Leevo Nov 30 '19 at 18:07
  • $\begingroup$ This regards standard convolutional neural networks with a fully connected layer at the output right? $\endgroup$ – Mattia Surricchio Nov 30 '19 at 18:10
  • $\begingroup$ Yes, but indirectly conv layers too I think, given that fully connected layers must receive, at some point, the signal from conv ones. If your image size changes, then the number of inputs of fully connected layers will change. It's true that conv filders "slide" on images of whatever size, but their relationship with fully connected layers must be "fixed" by the architecture. $\endgroup$ – Leevo Nov 30 '19 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.