0
$\begingroup$

Im currently trying to learn about back propagation, and it's going forward, but theres one thing that keeps me scratching my head, and doesnt really seems to be answered in any of the videos or articles im looking at. I understand now, that based on my loss, the weigths of my network is updated. But what i dont understand is how this happens. lets say i have this exercise network with the following weigths:

W_1 = 1.2 - w_2 = 0.4 - W_3 = 1.0

Now i do some training, and lets say i have the loss o.8. Now when i use my loss to update my weights, what happens specifically to the weights? are something being added, subtracted maybe multiplied?

Thanks a lot

Improve this question
$\endgroup$
1
  • $\begingroup$ Gradient Descent takes the derivative of the loss with respect to the weights, then uses that value to update the weights. In your example you haven't given us the formula to calculate the loss, saying "the loss is 0.8" is insufficient information to perform the gradient descent step. A formula for the loss is needed. $\endgroup$
    – A Kareem
    May 17, 2020 at 4:57

1 Answer 1

Reset to default
0
$\begingroup$

In short it is "Added" to previous value of the weight.

Here is the algo from tom mitchell's, in your case it shall be W_1 = w_1+delta,W_2 = w_2+delta, W_3 = w_3+delta.

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ nice, and delta is the gradient that have been propagated backwards? $\endgroup$
    – s3j80
    May 17, 2020 at 11:59
  • $\begingroup$ i wrote "Delta" for over simplification. its calculated in the equation T4.1. its the change in "weights" according to gradient to minimize the error/loss which needs to be updated. in case of back propagation yes updated backwards. $\endgroup$
    – BlackCurrant
    May 17, 2020 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.