0
$\begingroup$

I'm trying to understand the concept of receptive field better in the context of a practical CNN. All of the online info I can find on receptive field seems to be in a non-practical context so I'll ask my questions using this CNN as an example, which gets ~99% accuracy on MNIST:

class MnistNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 6, (5, 5))
        self.conv2 = nn.Conv2d(6, 16, (5, 5))
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
    # end function

    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), kernel_size=(2, 2))
        x = F.max_pool2d(F.relu(self.conv2(x)), kernel_size=(2, 2))
        x = torch.flatten(x, start_dim=1)      # flatten, except for the batch dimension
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        classificationLogits = self.fc3(x)
        return classificationLogits
    # end function
# end class

Here is how the layers break out:

enter image description here

Here is a drawing I did trying to understand how receptive field applies to this example:

enter image description here

Questions:

  1. Is my drawing correct? If not, where am I going wrong?

  2. When talking about receptive field, is it just understood that we talk about receptive field in the context of a single pixel in the last layer before flattening (so in this case that would be the 2nd max pool layer)?

  3. Ref #2 and my drawing above, in this case would the following statement be correct:

The receptive field of this net is 16 x 16

If this statement is not correct, how could it be changed to be correct?

  1. From the various sources I found on receptive field, it seems to be the general recommendation to choose net parameters so the receptive field covers the entire input image. Ref my drawing above the receptive field here only covers about 2/3 of the input image size, yet this net gets ~99% accuracy on MNIST. Does an undersized receptive field only work here because MNIST is a relatively simple task?
$\endgroup$

1 Answer 1

1
$\begingroup$

Your drawing looks correct (assuming a stride of 1 and no dilation). We can talk about the receptive field for any layer in the CNN - so the receptive field of your 1st conv2d is 5 x 5, and of your first pooling layer is 6 x 6. But a reference to the receptive field of the network is (as you said) the receptive field of the last layer before flattening.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.