I have been trying to train a model using WGAN loss functions, working different learning rates to choose my hyper parameters based on advice. I was told to try looking into keeping everything simple and handle my hyper parameters first. So I set my model to having all my layers with just one filter.
Batch size=64 and a latent dim=512 as they worked best.
When I try working with my learning rates though, I have tried using 0.005, 0.0005, 0.00005 and 0,000005
None of them seem to be working even remotely well.
My losses are being generated/ plotted as follows:
for epoch in range(epochs): start = time.time() disc_loss = 0 gen_loss = 0 for images in train_dataset: #images=(images-127.5)/127.5 #images=np.squeeze(images) images=np.expand_dims(images, axis=0) disc_loss += train_discriminator(images) a1.append(disc_loss) #a1[i]=disc_loss #a2[i]=gen_loss i=i+1 if disc_optimizer.iterations.numpy() % n_critic == 0: gen_loss += train_generator() a2.append(gen_loss)
and then plt.plot(a1) and a2.
For a learning rate of 0.0005 these are my losses
(Not sure of best way to upload)
The generator loss really bothers me as do the images. The other learning rates don't seem to be much better. Even when I beefed up my network (the heaviest layers having 512 filters, these problems persisted)
How do I work this out as no matter what I do, my models refuse to converge