0
$\begingroup$

During back propagation, the algorithm can modify the weight values or bias values to reduce the loss.

How does the algorithm decide whether it has to modify the weight values or bias values to reduce the loss?

Does it modify the weight values in one pass and bias values in another pass?

Thanks!

$\endgroup$
1
$\begingroup$

Actually, weight values and bias values are updated simultaneously in each pass of backpropagation. That’s because the orientation of loss gradient vector is determined by the partial derivatives of all weights and biases with respect to the loss function. So if in each pass, you want to move in the correct direction towards the minimun of loss function, you must update both weights and biases at the same time and in the correct orientation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.