Could you please explain in simplest way the algorithm (mathematical equation) of back-prop?

I read lot of articles, I know for what it is, and I understand the intuition behind it, but I still don't know the equation of "upgrading/changing" neurons' attributes in layers.

  • $\begingroup$ Could you please link or quote something which tries to explain what you want, but is not simple enough? Also, if you could explain which part you don't understand, or think is too complex, that would help. Otherwise, if I or someone else tries to answer you, we have to explain the whole thing, without being sure if it is simple enough or what you need. $\endgroup$ – Neil Slater Aug 9 '18 at 19:33
  • $\begingroup$ @NeilSlater I mean - how to calculate new weights for neurons - the general equation for it. $\endgroup$ – mikinoqwert Aug 10 '18 at 5:41
  • $\begingroup$ I mean please show me one or more articles that you have read (instead of just saying "lot of articles") and that you did not help you. You are asking for the "simplest explanation", so if I answer, I need to know what you think is complicated. I do not want to guess because I have not seen the articles. $\endgroup$ – Neil Slater Aug 10 '18 at 7:44
  • $\begingroup$ this is very simple as well as in-depth. : yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf $\endgroup$ – krishna Oct 9 '18 at 9:05

You can refer to this answer here for a very detailed explanation and derivation of the backpropagation algorithm.

In general, backpropagation is based on the idea that it is possible to attribute an amount of blame to each parameter in your model for the resulting error. So this is what we do, first we will set all the parameters in our neural network (weights and biases) randomly. Then we will pass examples through the model, and at the output we will compute some loss function. Then using backpropagation we will tune our model parameters proportionally to their contribution to the incurred error.

Here are some other related answers that explain the complexities of backpropagation:

| improve this answer | |
  • 1
    $\begingroup$ The equation "x_new=x_old−ν* dy/dx" in one of links above is, I think, exactly what I expected. Thanks! $\endgroup$ – mikinoqwert Aug 10 '18 at 6:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.