Recreating ResNet50

Question

I am attempting to recreate ResNet50 in Keras. I don't understand the process of creating a residual step in between blocks or even the process of creating blocks themselves. Is it as simple as:

Create the CNN

cnnModel = Model()

Block 1

cnnModel.add(Conv2D( kernel_size= (7,7), input_shape=(256,256,3), filters = 64, strides=2))

Block 2

cnnModel.add(MaxPool2D(pool_size=(3,3), strides=2))

Block 3

cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,)) cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,)) cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,)) cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,))

Block 4

cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,)) cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,)) cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,)) cnnModel.add(Conv2D( kernel_size= (3,3), input_shape=(256,256,3), filters = 64,))

....

Or do I have to create the "blocks" in a certain way?

I have also seen these "blocks" referred to as "stacks of layers"

Answer 1

I am afraid it is not that simple - Have a look at this pretty good walkthrough.

The table you posted is a kind of overview that doesn't contain all the details of how the "blocks" are linked. Other details such as max pooling after conv layers.

Your model therefore also doesn't yet contain the main idea, which is residual mappings, as shown by the black arrows jumping over several conv layers in this snippet of the diagram from the paper:

Here is the code for one of the blocks (taken from the blog I linked above):

def residual_block(y, nb_channels, _strides=(1, 1), _project_shortcut=False):
    shortcut = y

    # down-sampling is performed with a stride of 2
    y = layers.Conv2D(nb_channels, kernel_size=(3, 3), strides=_strides, padding='same')(y)
    y = layers.BatchNormalization()(y)
    y = layers.LeakyReLU()(y)

    y = layers.Conv2D(nb_channels, kernel_size=(3, 3), strides=(1, 1), padding='same')(y)
    y = layers.BatchNormalization()(y)

    # identity shortcuts used directly when the input and output are of the same dimensions
    if _project_shortcut or _strides != (1, 1):
        # when the dimensions increase projection shortcut is used to match dimensions (done by 1×1 convolutions)
        # when the shortcuts go across feature maps of two sizes, they are performed with a stride of 2
        shortcut = layers.Conv2D(nb_channels, kernel_size=(1, 1), strides=_strides, padding='same')(shortcut)
        shortcut = layers.BatchNormalization()(shortcut)

    y = layers.add([shortcut, y])
    y = layers.LeakyReLU()(y)

    return y

You can see they use the functional API (meaning they use Model instead of Sequential, from keras.models). Also note how the shortcut variable is stored at the beginning of the block as copy (a residual mapping) of the blocks input, which is then used to introduce the actual magic, gre-supply those residuals back to the network at the end of the block.

From the paper:

The operation F + x is performed by a shortcut connection and element-wise addition

which is basically what is going on after the if block in the line:

y = layers.add([shortcut, y])

The full code example at the end of the linked blog should help to give some further context.