# Backpropagation in a convolutional neural network with stride and padding

So i am trying to learn backpropagation of convolutional neural networks. A lot of articles only cover convolutions without a stride and a padding variable, so i decided to try it on my own. For simplicity i decided to try correlation (no filter flipping).
Symbols:

$$O$$ = output
$$W1$$ = output width
$$H1$$ = output height
$$I$$ = input
$$F$$ = filter
$$FS$$ = filter size (filter width and height are the same and equal to $$FS$$)

So I defined the correlation as:
$$\label{1}\tag{1} O_{i,j} = \sum_{x = 0}^{FS-1}\sum_{y = 0}^{FS-1}I_{x+i*s-p, y+j*s-p}*F_{x,y}$$

### Filter derivative:

$$\frac{\partial E}{\partial F_{i,j}} = \sum_{k=0}^{W1-1}\sum_{h=0}^{H1-1}\frac{\partial E}{\partial O_{k,h}}\frac{\partial O_{k,h}}{\partial F_{i,j}}$$ $$\frac{\partial O_{k,h}}{\partial F_{i,j}} = \frac{\partial }{\partial F_{i,j}}\left [ \sum_{x = 0}^{FS-1}\sum_{y = 0}^{FS-1}I_{x+k*s-p, y+h*s-p}*F_{x,y} \right ]$$ Derivative is non zero only when $$x = i$$ and $$y = j$$, therefore:
$$\frac{\partial O_{k,h}}{\partial F_{i,j}} = I_{i+k*s-p, j+h*s-p}$$ and
$$\label{2}\tag{2} \frac{\partial E}{\partial F_{i,j}} = \sum_{k=0}^{W1-1}\sum_{h=0}^{H1-1}\frac{\partial E}{\partial O_{k,h}}I_{i+k*s-p, j+h*s-p}$$

### Code attempt:

So i wanted to try if my calculations were correct, so i wrote this simple correlation and backpropagation in javascript: https://gist.github.com/jakic12/414ad450d9c1222e58d8e09c6b92cebb

First the forward propagation (correlation) - equation (1)

/**
* correlate an array a with a filter f
* @param{*} a the input array
* @param{*} f the filter
* @param{*} s stride
*/
function corre(a, f, s, p){
let outY = parseInt((a.length - f.length + 2 * p)/s + 1)
let outX = parseInt((a.length - f.length + 2 * p)/s + 1)

return new Array(outY).fill(0).map((_, y) =>
new Array(outX).fill(0).map((__, x) => {
let sum = 0;
for(let j = 0; j < f.length; j++){
for(let i = 0; i < f[j].length; i++){
if(a[y + j * s - p] && a[y + j * s - p][x + i * s - p])
sum += a[y + j * s - p][x + i * s - p] * f[y][x]
}
}
return sum
}
)
)
}


Then the error calculation (Mean squared error) and the partial derivatives with respect to the output layer ($\frac{\partial E}{\partial O_{k,h}}$)

/**
* Calculate the error and the partial derivatives with respect to the output layer
* @param{*} actual the actual output
* @param{*} exp the expected output
*/
function getError(actual, exp){
let err = 0
let out = new Array(actual.length).fill(0).map(() => new Array(actual.length))

for(let i = 0; i < out.length; i++){
for(let j = 0; j < out[i].length; j++){
out[i][j] = actual[i][j] - exp[i][j]
err += Math.pow(exp[i][j] - actual[i][j],2)
}
}
return { dO:out, err:err/2 }
}


Then calculating the derivatives with respect to the filter - equation (2)

/**
* backpropagate the correlation with given derivatives of the next layer
* @param{*} FS filter size
* @param{*} dO derivative with respect to the output of the correlation
* @param{*} input the input of the correlation
* @param{*} s stride
*/
function backpropFilter(FS, dO, input, s, p){
return new Array(FS).fill(0).map((_, j) =>
new Array(FS).fill(0).map((__, i) => {
let sum = 0

for(let h = 0; h < dO.length; h++){
for(let k = 0; k < dO[h].length; k++){
if(j+h*s-p >= 0 && j+h*s-p < input.length && i+k*s-p >= 0 && i+k*s-p < input.length)
sum += dO[h][k] * input[j+h*s-p][i+k*s-p]
}
}
return sum
}
)
)
}


### The problem

The problem is, that altho the network first starts descending, it then quickly starts going up and never stops: I am sorry, because i am not the best at derivatives, so i don't know where my problem is.
Help would be appreciated, thanks!

There is a typo in the code:

/**
* correlate an array a with a filter f
* @param{*} a the input array
* @param{*} f the filter
* @param{*} s stride
*/
function corre(a, f, s, p){
let outY = parseInt((a.length - f.length + 2 * p)/s + 1)
let outX = parseInt((a.length - f.length + 2 * p)/s + 1)

return new Array(outY).fill(0).map((_, y) =>
new Array(outX).fill(0).map((__, x) => {
let sum = 0;
for(let j = 0; j < f.length; j++){
for(let i = 0; i < f[j].length; i++){
if(a[y + j * s - p] && a[y + j * s - p][x + i * s - p])
sum += a[y + j * s - p][x + i * s - p] * f[y][x]
}
}
return sum
}
)
)
}

sum += a[y + j * s - p][x + i * s - p] * f[y][x]
^  ^


should be

sum += a[y + j * s - p][x + i * s - p] * f[j][i]
^  ^