0
$\begingroup$

I want to use ResNet50 as a feature extractor. For this purpose, I have loaded the pre-trained model, deleted a few layers and added my layers to the model. For adding my layers, I have used the Sequential API. The code is the following:

resnet_model = ResNet50(weights='imagenet')

# Delete layers
for i in range(12):
  resnet_model.layers.pop()

# Fix weights
for layer in resnet_model.layers:
  layer.trainable = False

base_model = Model(inputs=resnet_model.inputs, outputs=resnet_model.layers[-1].output)

model = Sequential()
model.add(base_model)

model.add(Conv2D(2048, kernel_size=(3, 3), input_shape=(7, 7, 2048)))
model.add(BatchNormalization(momentum=bn_momentum))
model.add(Activation('relu'))

model.add(Conv2D(2048, kernel_size=(3, 3)))
model.add(BatchNormalization(momentum=bn_momentum))
model.add(Activation('relu'))

model.add(Flatten())
model.add(Dense(1000, activation='relu'))
model.add(Dense(200, activation='softmax'))

This code works. However, when I run print(model.summary()) I obtain the following:

Model: "sequential_26"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
model_16 (Model)             (None, 7, 7, 2048)        19115904  
_________________________________________________________________
conv2d_31 (Conv2D)           (None, 5, 5, 2048)        37750784  
_________________________________________________________________
batch_normalization_23 (Batc (None, 5, 5, 2048)        8192      
_________________________________________________________________
activation_917 (Activation)  (None, 5, 5, 2048)        0         
_________________________________________________________________
conv2d_32 (Conv2D)           (None, 3, 3, 2048)        37750784  
_________________________________________________________________
batch_normalization_24 (Batc (None, 3, 3, 2048)        8192      
_________________________________________________________________
activation_918 (Activation)  (None, 3, 3, 2048)        0         
_________________________________________________________________
flatten_10 (Flatten)         (None, 18432)             0         
_________________________________________________________________
dense_17 (Dense)             (None, 1000)              18433000  
_________________________________________________________________
dense_18 (Dense)             (None, 200)               200200    
=================================================================
Total params: 113,267,056
Trainable params: 94,142,960
Non-trainable params: 19,124,096
_________________________________________________________________
None

As you can see, the conv layer adds 37750784 new parameters to the network!!! Obviously, this isn't right, but I can't see why this happens...

$\endgroup$
1
$\begingroup$

Actually, the number of parameters for the second layer is correct, the number of parameters in a convolutional layer is calculated as follows:

n_parameters = n_filters_l * (n_filters_l-1 * (kernel_size_h * kernel_size_w) + 1)
             = 2048 * (2048 * (3 * 3) + 1) = 37750784

If you want to decrease the number of parameters, you therefore have multiple options:

  • Decrease the number of filters in the layer before it
  • Decrease the number of filters in the layer itself
  • Use a lower kernel height of width
$\endgroup$
2
  • $\begingroup$ You're right. I didn't do the calculations right. However, then I don't know how there are so few parameters in all the other layers of the ResNet... $\endgroup$
    – Aeryan
    Nov 19 '19 at 21:34
  • $\begingroup$ After doing some research, it seems that what ResNet does is reducing the number of layers before performing the convolution and then do 1x1 convolutions to increase the number of features. This way, the number of parameters in the network doesn't explode. $\endgroup$
    – Aeryan
    Nov 19 '19 at 21:58

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .