I am running a GAN on the MNIST dataset. As the GAN continues to train, the quality of the generator seems to get worse, and it even seems as if it is starting to converge to just one value.
Epoch 7:
Epoch 21:
Why is this happening? I am running the code from this repo here: https://github.com/bstriner/keras-adversarial/blob/master/examples/example_gan.py
What you are experiencing is called mode collapse, which can occur in GAN Training and is one of its "training instability problems". If you are using Vanilla GAN one effective way is to implement mini batch discrimination, which for my understanding does give the discriminator batches of real and fakes and it has to decide batchwise if its fake or real. So if it encounters the same or nearly same image often it is easy for him to say it's fake. Other GAN Architecture like WGAN, and BEGAN state that they don't suffer from mode collapse which would be another alternative.
You can also try to expand the GAN to an AC-GAN, which produces classwise samples (0-9) but in AC-GAN it's also possible to suffer from classwise mode collapse.
Here is a pull request for keras implementing minibatch discrimination : https://github.com/keras-team/keras/pull/3677
