6
$\begingroup$

I'm trying to implement a convolutional neural network at the moment. A simple feedforward network is not the problem but I'm having some trouble with the weight adjustment in the conv layer.

Lets assume I have four layers. Input, convolution, hidden and output.

enter image description here

src:http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/

In the picture above we just see the input and the convolution layer. The deltas of the convolution layer are calculated as in a normal feedforward network. But how do I update the weights/filtermatrix between input and convolutionlayer?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

For learning kernel/filter matrix in convolution layer, we find partial derivative of loss w.r.t. filter matrix and use gradient descent method to update filters. $$ W = W - \alpha\frac{\partial L}{\partial W} $$

Convolutional Neural Networks also use back-propagation algorithm to find partial derivatives of loss w.r.t. filter matrix.

Improve this answer
$\endgroup$
4
  • $\begingroup$ This seems entirely correct to me, but it is not clear why the OP has not already understood this stage, since they have (also correctly) stated "the deltas of the convolution layer are calculated as in a normal feedforward network". I suspect as well as this answer, they are also missing the step of accumulating the deltas for each part of the feature output, so that they apply to the same weights each time. $\endgroup$
    – Neil Slater
    Feb 17, 2017 at 8:00
  • $\begingroup$ @avaj can u share link of your implementation? $\endgroup$
    – Bhagyesh Vikani
    Feb 17, 2017 at 8:31
  • $\begingroup$ I have a correct implementation right now. If somebody is interested anymore I can share this. $\endgroup$
    – avaj
    Apr 30, 2017 at 17:58
  • $\begingroup$ @avaj could you please share your implementation? $\endgroup$
    – Harsh Wardhan
    Jan 29, 2018 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.