# How the vector of weights assigned to a neural network?

You have set of input nodes, hidden nodes and output nodes although I have one output node in my case.

We assign weights to the connections between those nodes. So is there any pattern of assigning weights to those connections?

This is one pattern of assigning weights to neural network. See the another way of assigning the weights.

Please tell which one of the following is the correct way of assigning the weights to the neural network?

• – Nav
Mar 10, 2018 at 12:35

It looks like when you say "way of assigning the weights", that you mean "what order are weights counted in, so that I know which weights connect between which neurons".

There is no formal "correct" way of doing this for all neural networks. However, in practice for feed-forward networks like your diagram, you would choose to use a matrix, not a vector, to represent the weights connecting layers. That is how pretty much all standard libraries will represent weights.

A matrix uses two indices (call them $i,j$ in this case) to identify a single scalar value. If we call the weight matrix $W$, then an individual weight is $W_{ij}$.

To determine how the weights connect between neurons, then you index the input layer neuron with $i$ and the output layer neuron with $j$.

In math notation, looking at your diagram, call each input value $x_i$ and each hidden value $h_j$, then the formula for calculating a single $h_j$ would be:

$h_j = f(b_j + \sum_{i=1}^{N} W_{ij}x_{i})$

Where $f()$ is a transfer function (such as sigmoid), $N$ is the number of input features, and $b_j$ is the bias for hidden layer neuron $j$.

You will also often see this written using matrix notation:

$\mathbf{h} = f(W\mathbf{x} + \mathbf{b})$

. . . this is not only clear and simple notation, but using matrix maths like this to describe a neural network is what allows us to use high performance libraries on GPUs such as TensorFlow.

• Understood. But I am using R for training the neural network. So in that case how should we assign the weight matrix to the neural network? Can it be shown as to how the matrix of weight is written is assigned? Jun 1, 2017 at 10:16
• @Manik: R has built-in support for linear algebra including basics of matrix multiplication, so just use that. See statmethods.net/advstats/matrix.html Jun 1, 2017 at 10:25
• If the equation $h_j = f(b_j + \sum_{i=1}^{N} W_{ij}x_{i})$ is expressed using matrix notation, shouldn't it be: $\mathbf{h} = f(W^{\top}\mathbf{x} + \mathbf{b})$
– Joe
Jun 6, 2020 at 21:34

This might have more of what you are looking for...