I'm learning about convolutional neural networks. The convolution operation in order to extract features that is described in literature and posts used for this is quite intuitive and easy to understand (shown by the next gif), and even trivial to implement in a naive way:
I'm using scipy.signal.convolve2d instead of my own implementation for performance reasons. Checking the documentation, it mentions three different modes: full, valid and same. I've figured out, just by comparing results and shapes, that the valid mode corresponds to the operation the previous gif shows and that every place refers as "convolution". Also, that same is just the same, but with zero padding, i.e.:
In [1]: import numpy as np
In [2]: from scipy.signal import convolve2d
In [3]: image = np.random.rand(8, 8)
In [4]: padded_image = np.pad(image, (2, 2))
In [5]: kernel = np.random.rand(5, 5)
In [6]: assert (convolve2d(image, kernel, mode="same") == convolve2d(padded_image, kernel, mode="valid")).all()
The following gif would explain the operation in the same mode:
What I cannot find even an intuition for is the full mode (which the documentation refers as The output is the full discrete linear convolution of the inputs.). Using the same image and kernel, the operation gives:
In [7]: convolve2d(image, kernel).shape
Out[7]: (12, 12)
It's hard to get an understanding or juts an intuition by the result, and just by the description of the mode parameter and looking for literature about convolution operation. In the field of CNNs, the convolution is always explained as an operation to "reduce" the dimensions of an input image in order to extract its features.
This question comes from the need of having to propagate back the error delta between two convolutional layers. I need to calculate the error delta in layer l, so for that I'm performing a convolution between the kernel and the error delta of the l+1 layer (having then to multiply the gradients of layer l), and performing a valid convolution gives obviously an error delta whose dimensions don't match with the l layer activation, but it does doing a full convolution (figured out just by chance). I want to know if this is the correct way of propagating the error back.
