Why do most GAN (Generative Adversarial Network) implementations have symmetric discriminator and generator architectures?

Question

For example, if the discriminator is a vanilla network of n layers, each with n(i) units, then, typically, the generator will also be a vanilla network of n layers, each with n(n-i) units (except the output of the discriminator, where n(n) = 1 whereas for the generator n(0) = NOISE_SIZE).

If the discriminator is a CNN, the generator is typically a symmetric "deconvolution network" where the i-th layer is a transposed convolution layer, symmetric to the n-i-th layer of the discriminator.

Virtually all implementations I've seen follow this pattern, although I can't see anything in the theory why it would have to be the case. And yet, I had a simple vanilla implementation of a digit drawing GAN trained on MNIST, which worked reasonably well. I tried to improve the discriminator by making it a CNN (with the same architecture which I had working well on recognizing MNIST digits) without changing the generator. The GAN no longer worked and converged to a state where the generator always produced the same gibberish drawing. Intuitively, a better discriminator should help the GAN but it was not the case. It seems that improving the discriminator without equally improving the generator does not work (and vice-versa probably). Is this the reason why people always choose symmetric architectures? To preserve a "balance of skills" between the adversaries? Or is there a deeper reason?

Answer 1

The main reason is that you want the Generator and the Discriminator to be equally powerful. The base intuition of GANs is that the two Networks can improve themselves through competition. If one is way better than the other, they will get stuck into some unwanted equilibrium in which one beats the other all, or almost all the times.

For this reason, they are usually chosen symmetric. It's the simplest and most effective way to make sure their competition will remain balanced.

Answer 2

The reason, IMHO, that there is a similar pattern is more due to the convolutional layers than anything else. You can have an imbalance in the number of dense layers.

Let's take even the case of an autoencoder. If you take something simple like the SwissRoll, you will see that you don't need the same amount of layers on the encoder and the decoder side (basically your discriminator and the generator). But if you have an image, having only dense layers makes it hard for the generator to generate smooth images. It will take far more time to train it, and meanwhile, the discriminator can use this unsmoothness to detect them. Also, you have to remember that the C layers are preprocessors, extractors, the same can be said for the generator outputs, they are creators, they need to match.

So yes, it's about balancing, but only the features. The rest doesn't have to be symmetric, it's all about the complexity of the function you have to approximate.