I am trying to train an artificial neural network with two convolutional layers (c1, c2) and two hidden layers (c1, c2). I am using the standard backpropagation approach. In the backward pass I calculate the error term of a layer (delta) based on the error of the previous layer, the weights of the previous layer and the gradient of the activation in respect to the activation function of the current layer. More specifically the delta of layer l looks like this:
delta(l) = (w(l+1)' * delta(l+1)) * grad_f_a(l)
I am able to compute the gradient of c2, which connects into a regular layer. I just multiply the weights of h1 with it's delta. Then I reshape that matrix into the form of the output of c2, multiply it with the gradient of the activation function and am done.
Now I have a the delta term of c2 - Which is a 4D matrix of size (featureMapSize, featureMapSize, filterNum, patternNum). Furthermore I have the weights of c2, which are a 3D matrix of size (filterSize, filterSize, filterNum).
With these two terms and the gradient of the activation of c1 I want to calculate the delta of c1.
Long story short:
Given the delta term of a previous convolutional layer and the weights of that layer, how do I compute the delta term of a convolutional layer?