1
$\begingroup$

I have read a lot of papers and watched different videos, it seems like they explain how they are summed up with bias before entering the activation function.

What I am trying to understand is the whole flow process of data from the matrix all the way through to the output layer. What I am struggling with is how do they gain their initial weights?

I have drawn a diagram hoping to explain what I understand so far. Please let me know if I am in the right track and help me understand how weights are calculated. Many thanks in advance.

Many thanks in advance.

$\endgroup$
1
$\begingroup$

Initial weights are simply initialized by the programmer, usually according to some random distribution (e.g. Gaussian). There are several well known initializers, such as xavier, that have been shown to improve the training process.

If you read ML papers, you will see that researchers always mention how they initialized weights for their models so that their results are reproducible.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.