Suppose we have a network of neurons like below:
We make a little change in weight
w[l][j][k] on our network, and it can make change on our cost function from many paths
now we just pick one of that paths from our first choosed weight to -> it's activation's neuron to -> one of it's weights on the next layer to -> it's activation and so on...
we can imagine an equation (using chain rule) like below for this only one path and we call it a rate factor:
but it is only for one path of changing the cost as we changing that weight, so we need calculate the rate of cost changing by all possible related paths (all the rate factors) in the famous book of michael nielsen wrote that we calculating the rate of cost changing by doing sum all over the rate factors like below:
but really i don't understand exactly how can we calculate that slope of the cost with respect to that weight by summing all that rate factors how can relate that sum to our equation
a little later on that book we had a proof for an equation for the error in layer 'l' in terms of the error in the next layer, for that we have:
and here we do a sum over all related neurons errors too.
can someone give me a proof or an explain for that sum
how that rate of cost changing can relate to sum over all that related rate factors, why we use sum?
but using this for calculating that derivative is not so clear and tangible for me but any explanation about that sum over the derivatives can be very helpful for me.