3
$\begingroup$

I was reading about descending gradient. How does the descending gradient know what weights to adjust? Does it adjust to all network weights at the same time?

Does each weight have an associated error?

$\endgroup$

2 Answers 2

4
$\begingroup$

In general (or at least the basic implementation for the gradient descent), you apply for each iteration the update rule for each weight, using the partial derivative of the loss function with respect to that weight, as follows:

enter image description here

$\endgroup$
0
$\begingroup$

All the weights are updated while backpropagation in the gradient decent algorithm. The amount of weight update is determined by it gradient and the learning rate.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.