0
$\begingroup$

I was going through the relevant chain rule mathematics and I have successfully implemented backpropagation from scratch for MNIST (once, I even tried doing this for a small sample data I created by hand). I understand that the gradients form a chain and I can formulate this as convex optimization problem to get local minima.

But, is there any way to compute the error at each node? (I could not find any relevant mathematics; All of the University of Toronto material I checked uses the multiplication of previous error, derivative of activation layer and the input to compute derivative of error with respect to weights. This is then multiplied using a small learning rate.)

The reason I am asking is because of that derivative of error which is propagated backwards repeatedly. I am interested in computing the error of each node during an epoch/batch iteration.

Also, can you let me know why it is impossible (with relevant math) in case computation of error of each node is impossible?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.