I was going through the relevant chain rule mathematics and I have successfully implemented backpropagation from scratch for MNIST (once, I even tried doing this for a small sample data I created by hand). I understand that the gradients form a chain and I can formulate this as convex optimization problem to get local minima.

But, is there any way to compute the error at each node? (I could not find any relevant mathematics; All of the University of Toronto material I checked uses the multiplication of previous error, derivative of activation layer and the input to compute derivative of error with respect to weights. This is then multiplied using a small learning rate.)

The reason I am asking is because of that derivative of error which is propagated backwards repeatedly. I am interested in computing the error of each node during an epoch/batch iteration.

Also, can you let me know why it is impossible (with relevant math) in case computation of error of each node is impossible?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.