A fully-connected DNN has layer sizes of 3-3-4-2, where the first layer size represents the input layer. We assume that all layers are affine ones (no ReLU). Give the dimensions of all weight matrices and all bias vectors in the network and compute the total number of free parameters in this DNN.

According to this task the first layer size represents the input layer, so it must be 3.

a(0) = x = 3

If I know the input layer, so 3-4-2 are also the sizes of the bias vectors.

I know now the dimension of the input layer and the bias vector. The columns of W must be also 3, because of the size of the input layer. The rows of W must be equal to the size of the bias vector.

So I computed the following dimensions:

W(1) = W33, b3, a(1) = 3

W(2) = W43, b4, a(2) = 4

W(3) = W24, b2, a(3) = 2

But how to compute the total number of free parameters in this DNN?


1 Answer 1


Weights at first layer will depend on the Dimension of the input too.

Let's assume, This is your Network. Input has 5 Features ML Visuals by dair.ai $\hspace{6cm}$ Image credit - ML Visuals by dair.ai

Per layer weights and biases \begin{array} {|r|r|} \hline &Weights &Biases\\ \hline Layer-1 &5*3 = 15 &3\\ \hline Layer-2 &3*3 = 9 &3\\ \hline Layer-3 &3*4 = 12 &4\\ \hline Layer-4 &4*2 = 8 &2\\ \hline \end{array}

Check with Code

import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()

model.add(keras.layers.Dense( 3, activation="linear",input_shape=(5,)))
model.add(keras.layers.Dense( 3, activation="linear"))
model.add(keras.layers.Dense( 4, activation="linear"))
model.add(keras.layers.Dense( 2, activation="linear"))

#model.get_weights() #Can check weights matrix with this


enter image description here

  • $\begingroup$ Thank you! Afterwards this task wasn't bad at all. :) This was very helpful. $\endgroup$ Commented Jul 21, 2020 at 11:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.