4
$\begingroup$

enter image description here

Please explain how Z1 is working I just want to know why W is of shape (4,3) I understand that there are four Weights we are performing (4,3)*(3,1) + (4,1) but I don't understand what is 3 in (4,3)

Just write the full equation of Z1 rest is self-explanatory.

I New to This field so, please spare me its very common to forget basic concepts in beginnig

Improve this question
$\endgroup$
1
  • $\begingroup$ 3 is the number of xes $\endgroup$
    – user253751
    Jun 25, 2021 at 20:34

3 Answers 3

Reset to default
3
$\begingroup$

Neural Network is nothing but an accumulation of multiple Logistic Regressors. What is the Equation of Logistic Regression?

hypothesis = $W^T * x$ + b activation = Sigmoid (hypothesis)

Now, what does z1 tell us it tells us that first take the dot product between the input (x) and the weight matrix then apply the non-linearity. you have 3 input examples. $<x_1, x_2, x_3>$. and a weight matrix,

[[ $w^{1}_{11} , w^{1}_{12} , w^{1}_{13} , w^{1}_{14}$ ]

[ $w^{1}_{21} , w^{1}_{22} , w^{1}_{23} , w^{1}_{24}$ ]

[ $w^{1}_{31} , w^{1}_{32} , w^{1}_{33} , w^{1}_{34}$ ]],

Now you're multiplying the x with the transpose of W, i.e. $W^{t} * x$ Take the transpose of the Matrix W,

[[$w^{1}_{11} , w^{1}_{21} , w^{1}_{31}$]

[$w^{1}_{12} , w^{1}_{22} , w^{1}_{32}$]

[$w^{1}_{13} , w^{1}_{23} , w^{1}_{33}$]

[$w^{1}_{14} , w^{1}_{24} , w^{1}_{34}$]]

So, the input dimensions are (3, 1) Now the transpose matrix dimensions are (4, 3) and $W^T*x$ dimensions are (4, 3) * (3, 1) after multiplication the dimensions would be (4, 1). Now, Z1 is computed in this way.

The $W^1$ is telling the layer number means for which hidden layer this weight is initialized in your case your weight matrix is for hidden layer number 1. And the subscript $W_{11}$ or $W_{12}$...So on, represents the corresponding matrix multiplication indices $W_{1j}$ this first number in the subscript tells us the input number like in this case $x_1$. and the second subscript associated with the $W_{i2}$ tells us the neuron number in the corresponding hidden layer like in this case it is 2.

Now, lets compute $Z_1$,

$Z_1 = (w^{1}_{11} * x_1 + w^{1}_{21} * x_2 + w^{1}_{31} * x_2)$ + bias $ = sigmoid(Z_1)$ The rest are computed in the same and now you have the weight matrix and input vector you can compute the rest by yourself.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the effort I understood it now $\endgroup$
    – Shaelander Chauhan
    Jun 25, 2021 at 8:58
  • $\begingroup$ Welcome, then kindly thumbs up for the answer. $\endgroup$
    – Mohammad Ahmed
    Jun 25, 2021 at 9:02
1
$\begingroup$

First, welcome to the deep learning world. Take as simple as it is:

  • The dimensions of weights in any layer = (number of neurons in the next layer, number of neurons of the previous layer). In your example, the weights between the first hidden layer (4 neurons) and the input layer (3 inputs).
  • The dimensions of bias = (number of neurons of the connected layer, 1)
  • Z can be computed as in the following matrix: enter image description here

This is a step-by-step reference for your question: Using the right dimensions for your Neural Network

I hope that helps!

Improve this answer
$\endgroup$
1
$\begingroup$

Another way to look at it, visually, is that the fully connected layer is characterized by the set of edges between the inputs and the outputs, and each edge is represented by a single weight. In this particular construction, there are three inputs and four outputs, which gives a 4 x 3 matrix of weights.

In other words, you have four weights per input, and since there are three inputs, you have 4 x 3 weights.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.