2
$\begingroup$

I'm implementing a 1D CNN in keras by following the keras tutorial on the same - link. Once the model is built, when I execute model.summary(), I get the following output.

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 1000)              0         
_________________________________________________________________
embedding_1 (Embedding)      (None, 1000, 100)         17410600  
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 996, 128)          64128     
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 199, 128)          0         
_________________________________________________________________
conv1d_2 (Conv1D)            (None, 195, 128)          82048     
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 39, 128)           0         
_________________________________________________________________
conv1d_3 (Conv1D)            (None, 35, 128)           82048     
_________________________________________________________________
max_pooling1d_3 (MaxPooling1 (None, 1, 128)            0         
_________________________________________________________________
global_max_pooling1d_1 (Glob (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_2 (Dense)              (None, 20)                2580      
=================================================================
Total params: 17,657,916
Trainable params: 247,316
Non-trainable params: 17,410,600
_________________________________________________________________
None

The conv1d_1 has the total number of parameters as 64128. But since the conv1d_1 was initialized with filters = 128, kernel_size = 5, padding = 'valid' (which means no padding), shouldn't the number of parameters be

=> kernel_size * kernel_size * num_filters + num_filters * bias

=> 5 * 5 * 128 + 128 * 1

=> 26 * 128

=> 3328

$\endgroup$
1
  • $\begingroup$ embedding_1 (Embedding) (None, 1000, 100) 17410600 So how is the number of trainable parameters 17410600? I am not able to figure out the calculation for this. $\endgroup$
    – Manoj M
    Oct 26, 2018 at 1:24

2 Answers 2

3
$\begingroup$

In fact, you use 1D convolution. Given that the dimension of the output of embedding layer is 100, that the kernel size is 5, and that the number of filters is 128, You have 100x5x128 = 64000 weights. Add to this 128 biases and you get 64128 parameters.

Note that parameter sharing is used, so that there is only one set of weights and biases per filter, in depth.

$\endgroup$
0
$\begingroup$

$(5\times 100)+1$ no of parameters for the single filter, $+1$ is for bias term for each filter and we have 128 such filters so total no of parameters $= 501\times 128 =64128$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.