# How to propagate error back to previous layer in CNN?

I have a convolutional layer (link) with an input 5x5x2 (width, height, depth):

The layer has 3 filters with dimensions 3x3x2, it produces an output with dimensions 3x3x3.

I have completed the forward pass:

In a backward pass I calculated gradients of weights $dL/dw$, but I am not sure how to calculate $dL/dh$ (gradients for the previous layer).

First or all, is my calculation of $dL/dw$ valid ? (please refer to external spread sheet linked to the question) And most importantly, how do I propagate deltas to the previous layer ?

• Have a look at Jeremy Howard's Excel file hosted publicly on GitHub..He had simulated A CNN in EXCEL ITSELF.. – Aditya Mar 4 '18 at 16:46
• @Aditya Thank you, would you mind to share a link ? – koryakinp Mar 4 '18 at 17:12
• – Aditya Mar 4 '18 at 17:21
• @Aditya, thank you! It is very useful link, but unfortunately it does not cover backpropagation in CNN. – koryakinp Mar 4 '18 at 18:16
• What is $L$ in this question? – JahKnows Mar 7 '18 at 7:11

Since calculating the backward gradients that too in CNN is very math heavy and it's a bit different from normal MLP as in CNN'S we have Tensors not scalar values to multiply so People use Correlation to Calculate the Convolution Operation...

Just adding some links which I refer to from time to time thinking that someday I will be able to do this from scratch (seems impossible with deepening of Layers though...)

• Reference 1

• Reference 2

• CS231N Slides (Lecture 4-6 specially, the computational graphs construction are very helpful to understand the Back-propagation Algorithm..)

It would be great if Someone can transfer the Equations into Latex here So that the question can be answered formally..

Based on the above Links, here's what I got,(Came to know that Convolutions are equivalent to Correlation)

Image 1

Image 2

Stack doesn't allow me uploading image so using gdrive..(do rotate one of the images, the first one actually)

Hope it will give you an idea...

• I am a student so no sir please. really liked your idea of manually calculating the gradients, Thanks – Aditya Mar 10 '18 at 3:37
• Problem is solved? – Aditya Mar 12 '18 at 4:14
• Thank you, It is much clear for me now. If you ever visit Toronto, round of beer on me. – koryakinp Mar 12 '18 at 13:53