# Fully-Connected DNN: Compute the numbers of free parameter in a DNN

A fully-connected DNN has layer sizes of 3-3-4-2, where the first layer size represents the input layer. We assume that all layers are affine ones (no ReLU). Give the dimensions of all weight matrices and all bias vectors in the network and compute the total number of free parameters in this DNN.

According to this task the first layer size represents the input layer, so it must be 3.

a(0) = x = 3


If I know the input layer, so 3-4-2 are also the sizes of the bias vectors.

I know now the dimension of the input layer and the bias vector. The columns of W must be also 3, because of the size of the input layer. The rows of W must be equal to the size of the bias vector.

So I computed the following dimensions:

W(1) = W33, b3, a(1) = 3

W(2) = W43, b4, a(2) = 4

W(3) = W24, b2, a(3) = 2


But how to compute the total number of free parameters in this DNN?

Weights at first layer will depend on the Dimension of the input too.

Let's assume, This is your Network. Input has 5 Features $$\hspace{6cm}$$ Image credit - ML Visuals by dair.ai

Per layer weights and biases $$\begin{array} {|r|r|} \hline &Weights &Biases\\ \hline Layer-1 &5*3 = 15 &3\\ \hline Layer-2 &3*3 = 9 &3\\ \hline Layer-3 &3*4 = 12 &4\\ \hline Layer-4 &4*2 = 8 &2\\ \hline \end{array}$$

Check with Code

import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()