# In my GAN model, the discriminator loss quickly descends to magnitudes of $10^{-4}$ while generator loss is at levels of 5+?

I am creating a Generative Adversarial Network (GAN) for generating artificial trading cards, but I am a complete novice in the field. The problem I'm consistently having is that my discriminator, even though it is weaker (on the basis of learnable parameters), has its loss drop to magnitudes of $$10^{-4}$$ (ten to the power of negative four). In contrast, the generator loss accelerates from 5+ to 10+ in the first few epochs. Further, the discriminator's accuracy on both real and fake images instantly goes to 100%, varying at most by 2% on either end.

My current generative model:

def generator_model():

model = tf.keras.Sequential()

# First Dense Layer

# First Conv2DTranspose Layer
model.add(Conv2DTranspose(128, (5, 5), strides=(1, 1), padding='same', use_bias=False))

# Second Conv2DTranspose Layer
model.add(Conv2DTranspose(64, (5, 5), strides=(2, 2), padding='same', use_bias=False))

# Third Conv2DTranspose Layer
model.add(Conv2DTranspose(64, (5, 5), strides=(2, 2), padding='same', use_bias=False))

# Fourth Conv2DTranspose Layer
model.add(Conv2DTranspose(32, (5, 5), strides=(2, 2), padding='same', use_bias=False))

# Fifth Conv2DTranspose Layer
model.add(Conv2DTranspose(3, (3, 3), strides=(1, 1), padding='same', use_bias=False, activation='tanh'))

return model


My current discriminator model:

def discriminator_model():

model = tf.keras.Sequential()

# First Conv2D Layer
model.add(Conv2D(64, (5, 5), strides=(1, 1), padding='same', input_shape=[64, 64, 3]))

# Second Conv2D Layer

# Third Conv2D Layer

# Fourth Conv2D Layer

# Flatten the Output and Give Binary Output via Sigmoid Activation Function

optimizer = tf.keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5)

# Compile the Discriminator Model
model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])

return model


My current GAN model:

def gan_model(generator, discriminator):
GAN = tf.keras.Sequential()
discriminator.trainable = False

optimizer = tf.keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5)

GAN.compile(loss='binary_crossentropy', optimizer=optimizer)

return GAN


My current GAN training method: (I'm wondering if this is a terribly constructed training step)

def training_gan(gan_model, discriminator, generator, batch_size=256, epochs=100, epoch_steps=468, noise_dim=100):
# Training the model by enumerating epochs
for epoch in range(0,epochs):
for step in range(0, epoch_steps):
# Generating fake images
X_fake, y_fake = generate_img_using_model(generator, noise_dim, batch_size)
# Generating real images
X_real, y_real = generate_real_images(batch_size)
# Creating training set
X_batch = np.concatenate([X_real, X_fake], axis = 0)
y_batch = np.concatenate([y_real, y_fake], axis = 0)
# Training the discriminator
d_loss, _ = discriminator.train_on_batch(X_batch, y_batch)
# Gnerating noise input for the generator
X_gan = np.random.randn(noise_dim * batch_size)
X_gan = X_gan.reshape(batch_size, noise_dim)
y_gan = np.ones((batch_size, 1))
# Training the GAN model using the generated noise
gan_loss = gan_model.train_on_batch(X_gan,y_gan)
# Report the training progress
areport_progress(epoch=epoch, step=step, d_loss=d_loss, gan_loss=gan_loss, noise_dim=noise_dim, epoch_steps=epoch_steps)
# Report the progress on the full epoch
report_progress(epoch=epoch, step=step, d_loss=d_loss, gan_loss=gan_loss, noise_dim=noise_dim, epoch_steps=epoch_steps, gan_model=gan_model, generator=generator, discriminator=discriminator, eoe=True)


My current 'report_progress' method:

def report_progress(epoch, step, d_loss, gan_loss, noise_dim = None, epoch_steps= None, gan_model=None, generator=None, discriminator=None, n_samples=100, eoe= False):
if eoe and step == (epoch_steps-1):
# Report a full epoch training performance
# Sample some real images from the training set
X_real, y_real = generate_real_images(n_samples)
# Measure the accuracy of the discrinminator on real sampled images
_ , acc_real = discriminator.evaluate(X_real, y_real, verbose=0)
# Generates fake examples
X_fake, y_fake = generate_img_using_model(generator, noise_dim, n_samples)
# evaluate discriminator on fake images
_, acc_fake = discriminator.evaluate(X_fake, y_fake, verbose=0)
# summarize discriminator performance
# plot images
plt.figure(figsize=(20, 12), dpi=64)
for i in range(10 * 10):
# define subplot
plt.subplot(10, 10, 1 + I)
# turn off axis
plt.axis('off')
# plot raw pixel data
plt.imshow(upscale_image(X_fake[i, :, :, :])) #, cmap='gray_r')
#plt.show()
filename = 'generated_examples_epoch%04d.png' % (epoch+1)
plt.savefig(filename)
print('Disciminator Accuracy on real images: %.0f%%, on fake images: %.0f%%' % (acc_real*100, acc_fake*100))
# save the generator model tile file
filename = 'generator_epochs/generator_model_%04d.h5' % epoch
generator.save(filename)
filename = 'discriminator_epochs/discriminator_model_%04d.h5' % epoch
discriminator.save(filename)
filename = 'GAN_epochs/GAN_model_%04d.h5' % epoch
gan_model.save(filename)
elif step % 10 == 0:
# Report a single step training performance
print(f"[Epoch {epoch}, Step {step}] d_loss = {round(d_loss, 4)} | gan_loss = {round(gan_loss, 4)}")


Note: All of the above code is drawn mostly from a tutorial that I no longer have the link to readily available.

• Why, in so many tutorials, is 'use_bias' set to 'False' in all 'Conv2DTranspose' layers?
• Why is the 'tanh' activator used in the final 'Conv2DTranspose' layer?
• Why is 'discriminator.trainable' set to 'False' in the GAN model?

Any advice and/or recommended readings on the basics of building synergistic generative and discriminator networks would be much appreciated as well.

I can give you an answer regarding two of the three additional question. The reason that the activation for the last convolutional tranpose layer is set to tanh is most likely because the input data is also preprocessed to be in the (-1, 1) range, so it's making sure that both the real images and the generated images have a similar distribution. If the activation for the last layer would be different (e.g. a sigmoid activation), the ranges of the two distributions would differ quite a lot making it easy for the discriminator to determine if an image is real or fake.
Regarding the second question of setting the trainable attribute to False for the discriminator in the gan_model function, this seems to be related to allowing the discriminator to train when discriminator.train_on_batch is called but not when gan.train_on_batch for the combined model (see also this answer on a github issue)