0
$\begingroup$

I'm trying to implement a CNN, as part of an academic project to learn how it works. The project is a SRCNN: a convolutional neural network that increase the resolution of images.

Following this review Review: SRCNN (Super Resolution), the network includes 2 convolution layers with ReLu activation and 1 convolution layer without ReLu activation. Feed forwarding is quite simple. I'm having trouble on the backpropagation though.

Following this article CNNs, Part 2: Training a Convolutional Neural Network, I got the following derivative to update filters weights :

$$ \frac{\partial L}{\partial filter(x,y)}​​=\sum_{i=x}^{W-S+x}\sum_{j=y}^{H-S+y}\frac{\partial L}{\partial out(i,j)}*image(i+x,j+y) $$

where $L$ is the loss, $filter(x,y)$ the weight at the position $(x,y)$, $W$ the width of the image, $H$ the height of the image, $S$ the size of the filter.

But it does not take the activation function into account. It should work for the convolution layer without activation. But I can't see how to integrate ReLu derivative in this. And I didn't find any article that explains it clearly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.