9
$\begingroup$

In this highly cited paper, authors give the following discussion on the number of weight parameters. I am not very clear why it has $49C^2$ parameters. I think it should be $49C$ since each of $C$ input channels shares the same filter, which has $49$ parameters.

enter image description here

$\endgroup$
  • 1
    $\begingroup$ Using an image here reduces the quality of the question as it cannot be searched. Please consider using text instead. $\endgroup$ – Neil Slater Feb 21 '17 at 9:49
17
$\begingroup$

Actually it's $49C*C$, the first $C$ is the number of input channels, and the second $C$ is the number of filters.

Quote from CS231n:

To summarize, the Conv Layer:

  • Accepts a volume of size $W_1 \times H_1 \times D_1$
  • Requires four hyperparameters:
    • Number of filters $K$,
    • their spatial extent $F$,
    • the stride $S$,
    • the amount of zero padding $P$.
  • Produces a volume of size $W_2 \times H_2 \times D_2$ where:
    • $W_2 = (W_1 - F + 2P)/S + 1$
    • $H_2 = (H_1 - F + 2P)/S + 1$ (i.e. width and height are computed equally by symmetry)
    • $D_2 = K$
  • With parameter sharing, it introduces $F \cdot F \cdot D_1$ weights per filter, for a total of $(F \cdot F \cdot D_1) \cdot K$ weights and $K$ biases.
  • In the output volume, the $d$-th depth slice (of size $W_2 \times H_2$) is the result of performing a valid convolution of the $d$-th filter over the input volume with a stride of $S$, and then offset by $d$-th bias.

A common setting of the hyperparameters is $F = 3, S = 1, P = 1$. However, there are common conventions and rules of thumb that motivate these hyperparameters. See the ConvNet architectures section below.

$\endgroup$
  • $\begingroup$ Hi Icyblade, thanks for the reply. However, my question is still related to the statement "with parameter sharing, it introduces F.F.D1 weights per filter'. We know that each single filter has two connect to D1 input channels, so according to this statement, the weight connecting this filter is different among different input channels, is that right? $\endgroup$ – user297850 Feb 20 '17 at 23:46
  • $\begingroup$ The values of these weights are different, but the numbers are the same. $\endgroup$ – Icyblade Feb 21 '17 at 7:40
  • $\begingroup$ Using an image paste here reduces the quality of the answer as it cannot be searched. Please consider using text instead. However, I do agree with the answer - it seems correct to me. $\endgroup$ – Neil Slater Feb 21 '17 at 9:47
  • $\begingroup$ @user297850: Please do not extend your question - ask a new one if you still have problems. $\endgroup$ – Neil Slater Feb 21 '17 at 9:49
  • $\begingroup$ @NeilSlater Thanks for your tip, it takes me some time to reformatting. $\endgroup$ – Icyblade Feb 21 '17 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.