In ResNet papaer, First residual block's input size is 56x56x64 caused by 7x7x64 filter in first layer. But, in the paper, they showed residual block that has 56x56x256 input size. How does it is possible to change from 56x56x64 to 56x56x256?

enter image description here

  • $\begingroup$ can someone give an intuitive mathematically sound example of how this works? $\endgroup$ Dec 12 '20 at 15:20

I do not know the answer for sure but I assume that the "256-d" refers to the shortcut connection and not the input. Since the output dimension of the 3 conv. layers in your right hand picture (fig. 5) is 256 the shortcut is transformed from depth 64 to depth 256.

How this is being done is explained in the following paragraph on p. 4:

Residual Network. Based on the above plain network, we insert shortcut connections (Fig. 3, right) which turn the network into its counterpart residual version. The identity shortcuts (Eqn.(1)) can be directly used when the input and output are of the same dimensions (solid line shortcuts in Fig. 3). When the dimensions increase (dotted line shortcuts in Fig. 3), we consider two options: (A) The shortcut still performs identity mapping, with extra zero entries padded for increasing dimensions. This option introduces no extra parameter; (B) The projection shortcut in Eqn.(2) is used to match dimensions (done by 1×1 convolutions). For both options, when the shortcuts go across feature maps of two sizes, they are performed with a stride of 2.

For ResNet-34 this not required for the first building block which is shown in fig. 5 but for later blocks too. This is shown by the dotted lines in figure 3:

Figure 3. Example network architectures for ImageNet


Here 256-d can come from one of the three things termed in the paper as A, B, C.


A → Zero Padding Projections

B → projection shortcuts are used for increasing dimensions, and other shortcuts are identity.

C → all shortcuts are projections.

(as mentioned on page 6 of the paper)

In the Below Visualization of Resnet Block, a parallel 1x1 Operation is applied, which converts 64 filters to 256 filters.

One Resnet Block


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.