In my GAN, the discriminator loss goes down steadily, while the generator loss oscillates / does not converge.

I suspect this is due to the vanishing gradient problem. Theory: as the discriminator loss doesn't start out big to begin with, the generator will never be backpropagated small gradients into, and therefore it doesn't learn to generate better images.

What are some remedies to resolve this and make the generator actually work?

  • $\begingroup$ It would very helpful if you could share bits of your code $\endgroup$
    – Leevo
    Jan 31 '20 at 10:05
  • $\begingroup$ Can you share your architecture and loss function plus a plot of the loss curves? $\endgroup$
    – Sammy
    Mar 1 '20 at 20:58

This is one of the bigger problems with GANs, they are usually hard to train.

This is an active area of research and it does not seem like this problem has been solved. Without additional information or code, it is hard to help, but usually different loss functions are one way to mitigate the problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.