I was reading about descending gradient. How does the descending gradient know what weights to adjust? Does it adjust to all network weights at the same time?
Does each weight have an associated error?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
In general (or at least the basic implementation for the gradient descent), you apply for each iteration the update rule for each weight, using the partial derivative of the loss function with respect to that weight, as follows:
$\begingroup$
$\endgroup$
All the weights are updated while backpropagation in the gradient decent algorithm. The amount of weight update is determined by it gradient and the learning rate.
