1
$\begingroup$

As I was reading Y. LeCun's paper on Deep Learning (Nature, vol. 521, 2015), I came across a figure (the 1st one in the paper) which was associated to the backward pass during backpropagation through a convolutional network. I understood everything apart from the very last sentence of the legend that accompanied the figure: "Once the $\frac{∂E}{∂zk}$ is known, the error-derivative for the weight $w_{jk}$ on the connection from unit j in the layer below is just $y_j\cdot\frac{∂E}{∂zk}$".

It must be purely mathematical but I can't figure out why the error-derivative mentioned above equal $y_j$ times $\frac{∂E}{∂zk}$. Can someone explain to me this result?

Thx.

PS : link to the figure mentioned above is https://www.researchgate.net/figure/Multilayer-neural-networks-and-backpropagation-a-A-multi-layer-neural-network-shown-by_fig4_277411157

$\endgroup$
1
$\begingroup$

It would be helpful: https://www.youtube.com/watch?v=d14TUNcbn1k

As far as I understand, in backpropagation, we need to multiply the local gradient of each node by the gradient from the upper level and this is the value that we send back

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.