I'm working on neural networks. I have one very important question. For instance, there are two layers with 2 neurons in each. They are all fully connected. Which neurons do the weights belong to? Do they belong to neurons in next layer or the layer preceding it? Moreover, when weights are saved in Keras, are they saved for each neuron completely or they are saved as linknumber(first weight of every neuron)? I hope I'm able to clarify my question
2
-
2$\begingroup$ In Neuroscience or Machine Learning, Neurons(not in Neuro) don't hold anything, the connection between the neurons is what holds the weights. Simply, think of weights of how strong the connection b/n two neurons is. $\endgroup$– Kiritee GakSep 4, 2018 at 9:22
-
$\begingroup$ In mathematical terms, if you think of a neural network as a graph, with the neurons as vertices and the links between neurons as edges, the weights belong to the edges and not to the vertices. $\endgroup$– Adrian KeisterSep 4, 2018 at 11:32
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The neurons themselves only hold their activation values (or input values when we consider the input layer of neurons). The weights you mention actually indicate the strength of the connections between neurons in subsequent layers (and possibly the bias per layer). Using your terminology the weights would belong to the neurons in both the current and next layer.
For every neuron the strength of the connection to the other neurons is saved. In other words, all weights (and possibly biases) are saved.
For more information about the basics of neural networks, and more specifically multi-layer perceptrons, I can remmond this website.
