# Steps for back propagation of convolutional layer in CNN

Imagine I have the following layers in a CNN:

Conv-layer 1 (without reLU) [3 filters @ 1x3x3] => ReLU-layer 1 => Maxpooling-layer 1 [2x2]

Conv-layer 2 (without reLU) [10 filters @ 3x4x4] => ReLU-layer 2 => Maxpooling-layer 2 [2x2]

What I know is that, when "deriving" the Maxpooling-layer, the maximum value of that pooling has the value of 1 (because that value is the only one that is affecting the result of the CNN), and the other values take the value of 0.

After this step, I have the reLU-layer, where the derivative of the reLU-layer is 0 when x <= 0, and 1 when x > 0.

And in the last step, I have the Conv-layer, but I don't know how to calculate the derivative. How do I calculate the whole delta for that specific weight?

If there is a way to calculate the gradients of those weights using matrix notation, please let me know.

## Notation

The matrix notation that I use is the following: [number of filters - depth - rows - columns]. For example, when I say [3 filters - 1 x 3 x 3] I mean that I have 3 filters with depth 1, 3 rows and 3 columns (depth is "equivalent" to "channels").

Steps in Conv layer, using my notation

Imagine I have an input X with shape [depth:1, rows: 28, columns: 28]

Then, I have the conv-layer, where I have 3 filters (you can call the filter as W) with shape [depth:1, rows:3, columns:3]

So, what I do is, using 1 filter a time, convolve input X , so the resulting matrix has the following shape => [depth:1, rows: 26, columns: 26] (this is the result of convolving 1 filter). So, when I do the convolution using the 3 filters, the resulting matrix is shaped [depth:3, rows: 26, columns: 26].

After this, I apply ReLU to each value of the resulting matrix (the 3x26x26 matrix), and then apply non-overlapping maxpooling with a window of 2 rows x 2 columns.

Extra Note: The matrix notation that I use is the following: [number of filters - depth - rows - columns]. For example, when I say [3 filters - 1 x 3 x 3] I mean that I have 3 filters with depth 1, 3 rows and 3 columns (depth is "equivalent" to "channels").

Steps in Conv layer:

Imagine I have an input X with shape [depth:1, rows: 28, columns: 28]

Then, I have the conv-layer, where I have 3 filters (you can call the filter as W) with shape [depth:1, rows:3, columns:3]

So, what I do is, using 1 filter a time, convolve input X , so the resulting matrix has the following shape => [depth:1, rows: 26, columns: 26] (this is the result of convolving 1 filter). So, when I do the convolution using the 3 filters, the resulting matrix is shaped [depth:3, rows: 26, columns: 26].

After this, I apply ReLU to each value of the resulting matrix (the 3x26x26 matrix), and then apply non-overlapping maxpooling with a window o 2 rows x 2 columns.

If there is any doubt of what I explained, please comment me.

Note: I'm editing my own answer beacuse I don't have 50 reputation for doing comments.

• I know, the problem occured when I wrote the question and asked it as a guest, and not when logged in :( I was trying to edit the question but I can't Nov 24, 2016 at 20:22
• Now I have transferred this into the question, the site can delete your "answer" here. When I ask for this to happen, there might be an automated comment explaining the rule that is being applied. Nov 25, 2016 at 9:20