2
$\begingroup$

https://d2l.ai/chapter_computer-vision/transposed-conv.html

https://en.wikipedia.org/wiki/Transpose

I understand what transpose convolution does, but I am confused about the name of 'transpose'.

In linear algebra, transpose is the action which flips a matrix over its diagonal;, which is absolutely what transpose convolution does in this case.

So why named as transpose convolution?

$\endgroup$
1
  • 1
    $\begingroup$ Check This from 48:40 $\endgroup$ – 10xAI Apr 20 at 13:52
1
$\begingroup$

Transpose in mathematics means to change the order of matrix in an opposite way, the same notion carries here but not the exact sense, you are talking about.

The same problem exists with the word 'convolution', it means something else in mathematics. What is done in deep learning in name of convolution is cross-correlation in mathematics.

$\endgroup$
5
  • $\begingroup$ still doesn't explain why it's called transpose $\endgroup$ – physicsnoob1000 May 31 at 12:31
  • $\begingroup$ It is a misrepresentation as I said above. $\endgroup$ – Abhishek Verma Jun 1 at 8:42
  • $\begingroup$ It actually isn't. If you think of convolution as a sparse matrix multiplication, then transposed convolution is just the transposed matrix multiplication, see supplementary materials of arxiv.org/pdf/1602.05110.pdf $\endgroup$ – physicsnoob1000 Jun 2 at 11:49
  • $\begingroup$ You can suit your own narrative however you want. But the axiomatic definition of transpose (the transpose of a matrix is an operator which flips a matrix over its diagonal) doesn't match with what has happened. In a vague sense, it is related hence the reason it was named like that. $\endgroup$ – Abhishek Verma Jun 3 at 0:54
  • $\begingroup$ transposed matrix multiplication matches exactly what happens in transposed convolution, read the paper above, and watch the video (youtube.com/…) at 48:40 as suggested above, in particular the slide at 55:30 $\endgroup$ – physicsnoob1000 Jun 3 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.