2
$\begingroup$

I'm implementing a 1D CNN in keras by following the keras tutorial on the same - link. Once the model is built, when I execute model.summary(), I get the following output.

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 1000)              0         
_________________________________________________________________
embedding_1 (Embedding)      (None, 1000, 100)         17410600  
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 996, 128)          64128     
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 199, 128)          0         
_________________________________________________________________
conv1d_2 (Conv1D)            (None, 195, 128)          82048     
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 39, 128)           0         
_________________________________________________________________
conv1d_3 (Conv1D)            (None, 35, 128)           82048     
_________________________________________________________________
max_pooling1d_3 (MaxPooling1 (None, 1, 128)            0         
_________________________________________________________________
global_max_pooling1d_1 (Glob (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_2 (Dense)              (None, 20)                2580      
=================================================================
Total params: 17,657,916
Trainable params: 247,316
Non-trainable params: 17,410,600
_________________________________________________________________
None

The conv1d_1 has the total number of parameters as 64128. But since the conv1d_1 was initialized with filters = 128, kernel_size = 5, padding = 'valid' (which means no padding), shouldn't the number of parameters be

=> kernel_size * kernel_size * num_filters + num_filters * bias

=> 5 * 5 * 128 + 128 * 1

=> 26 * 128

=> 3328

$\endgroup$
1
  • $\begingroup$ embedding_1 (Embedding) (None, 1000, 100) 17410600 So how is the number of trainable parameters 17410600? I am not able to figure out the calculation for this. $\endgroup$ – Manoj M Oct 26 '18 at 1:24
3
$\begingroup$

In fact, you use 1D convolution. Given that the dimension of the output of embedding layer is 100, that the kernel size is 5, and that the number of filters is 128, You have 100x5x128 = 64000 weights. Add to this 128 biases and you get 64128 parameters.

Note that parameter sharing is used, so that there is only one set of weights and biases per filter, in depth.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.