# GAN Generator Backpropagation Gradient Shape Doesn't Match

In the TensorFlow example (https://www.tensorflow.org/tutorials/generative/dcgan#the_discriminator) the discriminator has a single output neuron (assume batch_size=1).

Then over in the training loop the generator's BinaryCrossentropy loss is calculated using the discriminator's output which has the shape [1]. It then calculates the loss gradients by plugging in the prediction and label into the dBinaryCrossentropy derivative whose resultant shape is also [1]. How is this [1] shaped gradient fed backward into the generator's layers when its shape doesn't match and the Conv2DTranspose layer expects gradients whose shape matches its output?

$$\frac{dL}{dZ} = \frac{dL}{dA}*\frac{dA}{dZ}$$ <--- the first term's shape is [1] but the second term's shape is the same as the Conv2DTranspose output shape, can't do hadamard product. How does backpropagation still work?

The gradient is backpropagated from the [1]-shaped loss back up through the whole discriminator and then through the generator, from its output back to its input.
• Yes, that's it, they form a single network (except when the discriminator is trained to recognize real images). My answer in grounded in the original GAN paper, which you linked, section 4 algorithm 1, at the parts that use $D(G(x))$ in the update of both discriminator and generator. As you can see, the discriminator receives the output of the generator.
• Can you confirm that my translation of algorithm 1 form the paper is correct in terms of the TF example? Run forward propagation on generator with noise to get a fake image to run it through discriminator, then run a real image through the discriminator. Calculate the combined loss gradients for these two and do backpropagation on discriminator only. Then run forward propagation for generator with noise again, run that generated image through discriminator, then calculate loss gradients. This is the [1] shaped gradient which is backpropagated through the generator + discriminator network. Jan 30, 2023 at 8:05