3
$\begingroup$

Imagine I have the following layers in a CNN:

Conv-layer 1 (without reLU) [3 filters @ 1x3x3] => ReLU-layer 1 => Maxpooling-layer 1 [2x2]

Conv-layer 2 (without reLU) [10 filters @ 3x4x4] => ReLU-layer 2 => Maxpooling-layer 2 [2x2]

What I know is that, when "deriving" the Maxpooling-layer, the maximum value of that pooling has the value of 1 (because that value is the only one that is affecting the result of the CNN), and the other values take the value of 0.

After this step, I have the reLU-layer, where the derivative of the reLU-layer is 0 when x <= 0, and 1 when x > 0.

And in the last step, I have the Conv-layer, but I don't know how to calculate the derivative. How do I calculate the whole delta for that specific weight?

If there is a way to calculate the gradients of those weights using matrix notation, please let me know.

Notation

The matrix notation that I use is the following: [number of filters - depth - rows - columns]. For example, when I say [3 filters - 1 x 3 x 3] I mean that I have 3 filters with depth 1, 3 rows and 3 columns (depth is "equivalent" to "channels").

Steps in Conv layer, using my notation

Imagine I have an input X with shape [depth:1, rows: 28, columns: 28]

Then, I have the conv-layer, where I have 3 filters (you can call the filter as W) with shape [depth:1, rows:3, columns:3]

So, what I do is, using 1 filter a time, convolve input X , so the resulting matrix has the following shape => [depth:1, rows: 26, columns: 26] (this is the result of convolving 1 filter). So, when I do the convolution using the 3 filters, the resulting matrix is shaped [depth:3, rows: 26, columns: 26].

After this, I apply ReLU to each value of the resulting matrix (the 3x26x26 matrix), and then apply non-overlapping maxpooling with a window of 2 rows x 2 columns.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

Extra Note: The matrix notation that I use is the following: [number of filters - depth - rows - columns]. For example, when I say [3 filters - 1 x 3 x 3] I mean that I have 3 filters with depth 1, 3 rows and 3 columns (depth is "equivalent" to "channels").

Steps in Conv layer:

Imagine I have an input X with shape [depth:1, rows: 28, columns: 28]

Then, I have the conv-layer, where I have 3 filters (you can call the filter as W) with shape [depth:1, rows:3, columns:3]

So, what I do is, using 1 filter a time, convolve input X , so the resulting matrix has the following shape => [depth:1, rows: 26, columns: 26] (this is the result of convolving 1 filter). So, when I do the convolution using the 3 filters, the resulting matrix is shaped [depth:3, rows: 26, columns: 26].

After this, I apply ReLU to each value of the resulting matrix (the 3x26x26 matrix), and then apply non-overlapping maxpooling with a window o 2 rows x 2 columns.

If there is any doubt of what I explained, please comment me.

Note: I'm editing my own answer beacuse I don't have 50 reputation for doing comments.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I know, the problem occured when I wrote the question and asked it as a guest, and not when logged in :( I was trying to edit the question but I can't $\endgroup$
    – John Kenis
    Nov 24, 2016 at 20:22
  • $\begingroup$ Now I have transferred this into the question, the site can delete your "answer" here. When I ask for this to happen, there might be an automated comment explaining the rule that is being applied. $\endgroup$
    – Neil Slater
    Nov 25, 2016 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.