1
$\begingroup$

If the discriminator and generator in GAN learn together, How the discriminator knows whether the image is real or fake in the initial phase of training? Does the discriminator need to get trained earlier?

$\endgroup$
1
$\begingroup$

The training regime of the discriminator is purely supervised: you give data and labels and it learns to predict the labels by minimizing the loss.

The only difference with the training of a normal classification network is that the distribution of one of the classes is not static but changes over time (as the generator gets better at creating real-looking data).

During the first training iterations, the generator will generate basically noise so, in the beginning, the discriminator will learn to tell real images apart from that noise. Then both will progressively improve together.

Depending on the adversarial loss being used (e.g. vanilla GANs), it may be important that the ability of generator and discriminator is not too imbalanced because if the discriminator learns "too much", the gradients propagated to the discriminator will be very small and the generator will not learn anything. In such cases, pretraining the discriminator may not be a good idea.

With Wasserstein GANs (or their gradient penalty-based variant) you should not have this problem so it should be possible to train the discriminator till optimality and use it to propagate gradients to the generator.

$\endgroup$
1
$\begingroup$

This a good question. I'll post my own personal considerations that should not be taken as rigorous answers, I'd like to broaden the question a little bit

[Disclaimer: personal considerations, not rigorous answer]

The generator and discriminator are not strictly learning together, they are learning one against other. Indeed, when the discriminator is training, the generator is frozen and vice versa.

At the very beginning of the training phase, the generated outputs of the generator are expected to be very far away from the real samples. It is also true that the discriminator is expected to not have a good feature space for the classification, but I think it gets to tell them apart faster than the generator begins to produce closer fake samples. Then, the minmax game evolves...

I thougth once to use a pre-trained classifier to force the generator produce real good fake images, but I also got to the feeling that, in that way, the battle will not be fair and will be biased to the discriminator side, regarding this:

  • Which input data should should be fed to the discriminator in its pre-training phase? I mean, to pre-train the discriminator, some pre-fake images are needed. One approach is to pre-train for a few steps the generator and produce pre-fake images which are far away from real images

  • Since the discriminator is still learning, are they chances it becomes even better?

  • The discriminator is so good it won't learn, is the game closer to nash equlibirum and so, closer to its end?

    Taking into account the enormous problems when training GANs, I've never tried it out.

It would be fantastic if more experienced GANners offer more insight into these considerations

$\endgroup$
1
  • $\begingroup$ Yes, Thanks for your explanation $\endgroup$ – Suresh Pokharel Jul 5 '19 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.