My question is regarding the transposed convolution operation (also commonly called deconvolution or upconvolution). In TensorFlow, for instance, I refer to this layer.
My question is, how / when do we add the bias (intercept) term when applying this layer?
When working with 'regular' convolution, we do this:
conv_output = tf.nn.conv2d(input, kernel, strides, padding='VALID') conv_output = tf.nn.bias_add(conv_output, bias)
How do we do this when applying the deconvolution layer? My confusion arises because my advisor told me to visualise upconvolution as a pseudo-inverse convolutional layer (inverse in the sense that convolution downsamples the input, while transposed convolution upsamples it. I know they are not mathematically inverse.)
According to him:
conv = x.w + b
x = (conv - b).W (where w and W are not the same).
Is the above equation even right? Something about it makes me feel uneasy.
In this scenario, since we are "going backwards", should we do something like this:
deconv_output = tf.nn.bias_add(input, -1 * bias) deconv_output = tf.nn.conv2d_transpose(deconv_output, kernel, strides, padding='VALID')
Or should we add the bias after applying the transpose convolution, as we do in 'regular' convolution?