Adjusting weights in an convolutional neural network

I'm trying to implement a convolutional neural network at the moment. A simple feedforward network is not the problem but I'm having some trouble with the weight adjustment in the conv layer.

Lets assume I have four layers. Input, convolution, hidden and output.

In the picture above we just see the input and the convolution layer. The deltas of the convolution layer are calculated as in a normal feedforward network. But how do I update the weights/filtermatrix between input and convolutionlayer?

1 Answer

For learning kernel/filter matrix in convolution layer, we find partial derivative of loss w.r.t. filter matrix and use gradient descent method to update filters. $$W = W - \alpha\frac{\partial L}{\partial W}$$

Convolutional Neural Networks also use back-propagation algorithm to find partial derivatives of loss w.r.t. filter matrix.

• This seems entirely correct to me, but it is not clear why the OP has not already understood this stage, since they have (also correctly) stated "the deltas of the convolution layer are calculated as in a normal feedforward network". I suspect as well as this answer, they are also missing the step of accumulating the deltas for each part of the feature output, so that they apply to the same weights each time. – Neil Slater Feb 17 '17 at 8:00
• @avaj can u share link of your implementation? – Bhagyesh Vikani Feb 17 '17 at 8:31
• I have a correct implementation right now. If somebody is interested anymore I can share this. – avaj Apr 30 '17 at 17:58
• @avaj could you please share your implementation? – Harsh Wardhan Jan 29 '18 at 12:47