How to choose the number of output channels in a convolutional layer?

Question

I'm following a pytorch tutorial where for a tensor of shape [8,3,32,32], where 8 is the batch size, 3 the number of channels and 32 x 32, the pixel size, they define the first convolutional layer as nn.Conv2d(3, 16, 5 ), where 3 is the input size, 16 the output size and 5 the kernel size and it works fine.

in_size = 3
hid1_size = 16
hid2_size = 32
out_size = len(labels)
k_conv_size = 5 

class ConvNet(nn.Module):

    def __init__(self):
        super(ConvNet, self).__init__()

        self.layer1 = nn.Sequential(
            nn.Conv2d(in_size, hid1_size, k_conv_size ),
            nn.BatchNorm2d(hid1_size),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2))

        self.layer2 = nn.Sequential(
            nn.Conv2d(hid1_size, hid2_size, k_conv_size),
            nn.BatchNorm2d(hid2_size),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2))

        self.fc = nn.Linear(hid2_size *  k_conv_size * k_conv_size, out_size)

    def forward(self, x):
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.fc(out)

        return out

I change the output size from 16 to 32 and that of the next layer from 32 to 64 and it still works. But when I resize the tensor to have the shape [8, 3, 64, 64], it throws a mismatch error that says size mismatch, m1: [16 x 5408], m2: [800 x 4] I understand m2 is what it's expecting and m1 is what I'm giving.

But I don't understand where the values of m2 and m1 come from and how to change the hid1_size accordingly.

I understand the relationship between the shape of input data and the neurons in the first layer when building regular linear layers but how to define the relationship between the shape of the input and the number of channels produced by the convolutional layer in cnns?

Answer 1

I recommend you reading the guide to convolution arithmetic for deep learning . There you can find very well written explanations about calculating the about size of your layers depending on kernel size, stride, dilatation, etc.

Further you can easily get your intermediate shapes in pytorch by adding a simple print(x.shape) statement in your forward pass and adapting your number of neurons in your fully connected layers.

Last but not least. When you cange your input size from 32x32 to 64x64 your output of your final convolutional layer will also have approximately doubled size (depends on kernel size and padding) in each dimension (height, width) and hence you quadruple (double x double) the number of neurons needed in your linear layer.