2
$\begingroup$

I am following this paper. You don't have to read it all as I'm about to explain the best I can. Consider the network that they have developed:enter image description here

This network takes as an input a vector of 10 values. Input is fed to convolution layer. Conv layer has a total of 384 filters of three different types. By type here I mean filter size. 128 filter for each size. Sizes are 3,4 and 5. Note that these are all vectors and not matrices. Total of 384 feature vectors are produced as a result. These feature vectors are fed into Relu and then maxpool. Maxpool takes only one max value from the feature vector. After maxpool layer I have 384 vector and this is then fed into FC layer. FC layer has 384 neurons. And then output layer which has 3 neurons (for three classes) and a softmax applied to it.

Backpropogation:

To update weights of output layer, I have used following equation

Wnewij = Woldij - (learning_rate * (ActualValueOfThatNeuron - ExpectedValueOfThatNeuron)*Inputi

Where i is the input neuron in FC layer to j neuron in output layer.

To calculate gradient/error/loss to be propogated back (from FC layer neurons), I have used the following equation:

ErrorInputi = ( (ActualValueOfThatNeuron - ExpectedValueOfThatNeuron)*Wij)

(By input here I just mean a neuron in FC layer Then to calculate the weights from input to FC layer, I use the same equation as stated above except I use ErrorInputij like this:

Wnewij = Woldij - (learning_rate * (ErrorInputij)*Inputi

And then I computed Loss/gradient/error to be propogated back the same way. And now I am stuck at Maxpool (though Im not even sure about the above approach either). For the maxpool, gradient at max is one and for the rest zero. In this case, just considering one neuron, My vector might look something like this [0 loss 0 0 0 0]. Am I supposed to multiply it with input feature vector? How do I backpropogate error further down?

Complete newbie, please keep the mathematical equations to a minimum.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.