You have set of input nodes, hidden nodes and output nodes although I have one output node in my case.

We assign weights to the connections between those nodes. So is there any pattern of assigning weights to those connections?

enter image description here

This is one pattern of assigning weights to neural network. See the another way of assigning the weights.

enter image description here

Please tell which one of the following is the correct way of assigning the weights to the neural network?


2 Answers 2


It looks like when you say "way of assigning the weights", that you mean "what order are weights counted in, so that I know which weights connect between which neurons".

There is no formal "correct" way of doing this for all neural networks. However, in practice for feed-forward networks like your diagram, you would choose to use a matrix, not a vector, to represent the weights connecting layers. That is how pretty much all standard libraries will represent weights.

A matrix uses two indices (call them $i,j$ in this case) to identify a single scalar value. If we call the weight matrix $W$, then an individual weight is $W_{ij}$.

To determine how the weights connect between neurons, then you index the input layer neuron with $i$ and the output layer neuron with $j$.

In math notation, looking at your diagram, call each input value $x_i$ and each hidden value $h_j$, then the formula for calculating a single $h_j$ would be:

$h_j = f(b_j + \sum_{i=1}^{N} W_{ij}x_{i})$

Where $f()$ is a transfer function (such as sigmoid), $N$ is the number of input features, and $b_j$ is the bias for hidden layer neuron $j$.

You will also often see this written using matrix notation:

$\mathbf{h} = f(W\mathbf{x} + \mathbf{b})$

. . . this is not only clear and simple notation, but using matrix maths like this to describe a neural network is what allows us to use high performance libraries on GPUs such as TensorFlow.

  • $\begingroup$ Understood. But I am using R for training the neural network. So in that case how should we assign the weight matrix to the neural network? Can it be shown as to how the matrix of weight is written is assigned? $\endgroup$
    – Manik
    Jun 1, 2017 at 10:16
  • $\begingroup$ @Manik: R has built-in support for linear algebra including basics of matrix multiplication, so just use that. See statmethods.net/advstats/matrix.html $\endgroup$ Jun 1, 2017 at 10:25
  • $\begingroup$ If the equation $h_j = f(b_j + \sum_{i=1}^{N} W_{ij}x_{i})$ is expressed using matrix notation, shouldn't it be: $\mathbf{h} = f(W^{\top}\mathbf{x} + \mathbf{b})$ $\endgroup$
    – Joe
    Jun 6, 2020 at 21:34

This might have more of what you are looking for...

Building a neural network from scratch in R Tea & Stats with David Selby.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.