How are weights calculated in a feed-forward neural network before they are summed up with bias?

Question

I have read a lot of papers and watched different videos, it seems like they explain how they are summed up with bias before entering the activation function.

What I am trying to understand is the whole flow process of data from the matrix all the way through to the output layer. What I am struggling with is how do they gain their initial weights?

I have drawn a diagram hoping to explain what I understand so far. Please let me know if I am in the right track and help me understand how weights are calculated. Many thanks in advance.

Many thanks in advance.

Answer 1

Initial weights are simply initialized by the programmer, usually according to some random distribution (e.g. Gaussian). There are several well known initializers, such as xavier, that have been shown to improve the training process.

If you read ML papers, you will see that researchers always mention how they initialized weights for their models so that their results are reproducible.