1
$\begingroup$

Considering a hypothetical scenario , where we have 10 input layers, and 5 output layers.

How many weights are there in the neural network?

If this is implemented in pytorch, the answer will be 50. But shouldn't it be 15 since 10 weights from input layer, and 5 weights from output layer? Please resolve my doubt.

Please forgive my english, not a native speaker.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

All units in one layer gets connected to all units in the next layer. To accomplish this all units in the first layer will need a weight per unit in the last layer. Therefore you get the number of weight be multiplication and not addition like you assumed.

Let's for example add one more layer with 3 units in the end and maybe it will be clearer. We have layers with units [10, 5, 3]. Then we will have $W_1: 10\times5=50$ weights between the first two layers and $W_2: 5\times3=15$ between the last two layers and together they will be $W_1 + W_2 = W <=> 50 + 15 = 65$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your reply, Suppose layer one has W1 weights, and layer two has W2 weights, then should the trainable weights be W1 + W2? I am really sorry, but I am quite not able to understand your answer. Can you give me some more insights, I am stuck here and kinda desperately need them. $\endgroup$
    – TheMightyBarbarian
    Commented Feb 15, 2019 at 9:34
  • $\begingroup$ I think perhaps your are confusing units in a layer with the weights in a layer. One unit will be associated with several weights. When you say input layer has 10 and and output layer has 5, then you are talking about units and not about weights. The weights are the connection between units in two layers, so in your example it would just be one set of weights. If your draw 10 dots in a column and then 5 dots in another column, draw all possible lines between the columns and you will have 50 lines, those are your weights. $\endgroup$
    – Simon Larsson
    Commented Feb 15, 2019 at 11:50
  • $\begingroup$ Your first layer has no weights on its' own. $\endgroup$
    – Simon Larsson
    Commented Feb 15, 2019 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.