1
$\begingroup$

Note: This is an academics based problem.

So in a recent in-class quiz, we were asked that if we have an input layer consisting of 20 nodes along with 2 hidden layers (one of size 10 and the other of size 5), what will the total number of parameters in this network? How can we compute this?

Additionally, how do we know what shapes are they weights of? How can we determine which activation functions are suitable for such a neural network?

My idea was that (20*10) + (10*5) + (biases = 10+5) = 265. So 265 should be the number of parameters. For shapes/activation functions, from what I understand, it just depends on the data, no? Couldn't think of any way to directly predict it from this limited information

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ What do you think the answer should be? $\endgroup$
    – Akavall
    Commented Dec 23, 2019 at 0:56
  • 1
    $\begingroup$ @Akavall My idea was that (20*10) + (10*5) + (biases = 10+5) = 265. So 265 should be the number of parameters. For shapes/activation functions, from what I understand, it just depends on the data, no? Couldn't think of any way to directly predict it from this limited information. $\endgroup$
    – x89
    Commented Dec 23, 2019 at 5:15
  • $\begingroup$ I am not an expert, but as far as I can tell, there is not enough information in the question to answer the part about activation functions. For hidden layers, ReLU seems to be default choice, but Tanh and Sigmoid could also be fine, the best way is to try and see. For output layer, you could answer it (linear for regression, sigmoid for binary classification, softmax for multi-class problem), but the question does not specify the type of output being generated. $\endgroup$
    – Akavall
    Commented Dec 23, 2019 at 22:27

1 Answer 1

Reset to default
0
$\begingroup$

Actiavation function isnt a parameter.

But here is general formula for counting weghts:

Suppose for neural network with two hidden layers, inputs dimension is "I", Hidden number of neurons in Layer 1 is "H1", Hidden number of neurons in Layer 2 is "H2" And number of outputs is "O"

weights = (I+1)*H1 +(H1+1)*H2 +(H2+1)*O

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, I didn't mean that activation function is a parameter. I don't have any information regarding the outputs in this case. Then how can I use this formula? $\endgroup$
    – x89
    Commented Dec 23, 2019 at 9:01
  • $\begingroup$ towardsdatascience.com/… I tried following example 1.2 but again, I don't have the output so I don't know how to compute it. $\endgroup$
    – x89
    Commented Dec 23, 2019 at 9:02

Not the answer you're looking for? Browse other questions tagged or ask your own question.