What does depth mean in the SqueezeNet architectural dimensions table?

First time reading the SqueezeNet paper. Based on my understanding, a fire module contains a squeeze layer of 1x1 filters and a expand layer of 1x1 and 3x3 filters. If we take fire2 for instance, the input dimension is 55x55x96 and we take 16 1x1 filters to convolve over it. This returns a 55 x55x16 output. We then take the output and apply two convolutions, one with 64 1x1 filters and the other with 64 3x3 filters. We then concatenate the two results to create a final output of 55x55x128. In this case, what does the depth of 2 mean? Also, how do I calculate the # of parameters for each layer? In this paper the depth is defined as the number of layers. The table shows a depth=2 for the fire layers because each is comprised of 2 layers. First is the squeeze layer and then followed by the expand layer.

To calculate the number of parameters of a CNN we can do as follows for a single layer. Assume an input that is 28*28*64. This is the size of the MNIST dataset with 64 channels. Now assuming we want to convolve this input with 32 filters of a 3*3 kernel.

Let us give the following variable names, $l=64$ is the number of channels in the input, $m=n=3$ is the size of the kernel and $k = 32$ is the number of filters. The number of parameters is then calculated as

$((n*m*l)+1)*k = ((3*3*64)+1)*32 = 18,464$.

The $+1$ is used to add the biases.

To calculate the number of parameters in fire2 we first note that this module is comprised of 2 distinct layers.

Layer 1 - The squeeze layer - $s_{1x1}$

$l = 96$, $m = n = 1$ and $k = 16$. Thus giving a total of 1,552.

Layer 2 - The expansion layer

Kernel size 1 - $e_{1x1}$

$l = 16$, $m = n = 1$ and $k = 64$. Thus giving a total of 1,088.

Kernel size 3 - $e_{3x3}$

$l = 16$, $m = n = 3$ and $k = 64$. Thus giving a total of 9,280.

Total

Thus, a total of $1,552 + 1,088 + 9,280 = 11,920$.

• Thank you! I see that they merge the two expand layers after convolution. Are they just stacking the two outputs along the channel dimension? Does it matter which comes first? Jun 14 '18 at 14:59
• No it does not matter because the weights will just train accordingly! Jun 15 '18 at 0:46