1
$\begingroup$

I was reading A Guide to Convolutional Arithmetic to understand Transpose Convolution as it is cited in Keras and Theano documentation. I am having trouble understanding the following two statements :

From section 4.1

Using this representation, the backward pass is easily obtained by transposing C; in other words, the error is backpropagated by multiplying the loss with C.T. This operation takes a 4-dimensional vector as input and produces a 16-dimensional vector as output, and its connectivity pattern is compatible with C by construction.

When I try this out in pytorch, the error is certainly not equal to multiplying with C.T

import torch
import torch.nn.functional as F

x = torch.arange(1, 17, dtype=torch.float).resize_(4, 4)
w = torch.rand(3, 3)

Convolve w and x

# Convert x into an "image" tensor with a single channel and part of a mini-batch of size 1
x1 = x.view(1, 1, 4, 4)
x1.requires_grad = True

# Convert w into a conv filter with a single input channel and a single output channel
w1 = w.view(1, 1, 3, 3)
w1.requires_grad = True

y1 = F.conv2d(x1, w1)

Backpropagate

y1.backward(torch.ones_like(y1))
x1.grad

Now create the C matrix as mentioned in the paper.

C = torch.zeros(4, 16, dtype=torch.float)

C[0, :3] = w[0]
C[0, 4:7] = w[1]
C[0, 8:11] = w[2]

C[1, 1:4] = w[0]
C[1, 5:8] = w[1]
C[1, 9:12] = w[2]

C[2, 4:7] = w[0]
C[2, 8:11] = w[1]
C[2, 12:15] = w[2]

C[3, 5:8] = w[0]
C[3, 9:12] = w[1]
C[3, 13:] = w[2]

Multiplying unrolled y1 by C.T will not equal to x1.grad.

torch.mm(C.transpose(0, 1), y1.view(-1, 1)).view(4, 4)

Where is my mis-understanding?

And the second statement that I have not been able to verify has been cited in a lot intuitive answers on StackExchange/Quora, etc.

Section 4.3

Another way to obtain the result of a transposed convolution is to apply an equivalent – but much less efficient – direct convolution. The example described so far could be tackled by convolving a 3 × 3 kernel over a 2 × 2 input padded with a 2 × 2 border of zeros using unit strides (i.e., i' = 2, k' = k, s' = 1 and p' = 2), as shown in Figure 4.1. Notably, the kernel’s and stride’s sizes remain the same, but the input of the transposed convolution is now zero padded.

When I try this in pytorch, the two results are not equivalent except for having the same shape.

F.conv_transpose2d(y1, w1)
F.conv2d(y1, w1, padding=2)

Again, I am pretty sure I am mis-understanding something, but can't put my finger on what.

Any help in clarifying these concepts greatly appreciated.

P.S: I have already taken a look at this question and it does not answer my question.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.