0
$\begingroup$

I am training a Vanilla-GAN(or original GAN 2016) on a pokemon dataset https://www.kaggle.com/kvpratama/pokemon-images-dataset, for few epochs the discriminator has 100% accuracy over the real examples, but as the epochs pass it falls to 0% around 4-5 epochs.

One interesting effect however is that when I increase my batch size from 32 to 64, this effect is seemed to have a delayed effect. i.e It kicks in around 400-500 epochs.

Please help.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

I figured out my answer a while back. SO here it goes...

  1. Ensure that your training data is normalized. This was what causing the discriminator's accuracy to fall.
  2. You need to decrease your dropouts as it can cause heavy bias if done ineffectively.
  3. Remove any unnecessary layers in your discriminator as they can cause overfitting.
  4. Use batch normalization.
  5. Introduce dropouts in your generator.
Improve this answer
$\endgroup$
0
$\begingroup$

The discriminator gets trained two times at once, with real images and with the generator's fake output. Consequently, $err_{real}+err_{fake}$ gets backpropagated. The generator gets, trained by fooling discriminator to classify it as real.

However, the point to note is that they do not get trained on all the data at once, instead, get trained batch-wise. Which means intuitively, greater the batch_size, better is the discriminator(especially at the beginning) since it gets trained on more real images at once. Therefore, it takes much more time for the generator to fool the discriminator resulting in slowing the already difficult convergence.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi,thanks for the help. I recently found my answer, and it would help others to identify the problem. As you said, the bigger batch_size means better discriminator. However this is not true. As to contradict this, you can train a classifier on a very large set of images all labeled to 1, and then when you try it to train on some other set of images say label 0. It will fail to train itself, because of all the over-fitting it does. Smaller batch sizes includes a higher delta of variety with each batch effectively training both discriminator, and generator. $\endgroup$
    – TheMightyBarbarian
    Jul 23, 2019 at 5:49
  • $\begingroup$ There are some exceptions of very small batch sizes. Like batch_size < 32 $\endgroup$
    – TheMightyBarbarian
    Jul 23, 2019 at 5:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.