0
$\begingroup$

I have been trying to train a model using WGAN loss functions, working different learning rates to choose my hyper parameters based on advice. I was told to try looking into keeping everything simple and handle my hyper parameters first. So I set my model to having all my layers with just one filter.

Batch size=64 and a latent dim=512 as they worked best.

When I try working with my learning rates though, I have tried using 0.005, 0.0005, 0.00005 and 0,000005

None of them seem to be working even remotely well.

My losses are being generated/ plotted as follows:

for epoch in range(epochs):
        start = time.time()
        disc_loss = 0
        gen_loss = 0
        for images in train_dataset:
            #images=(images-127.5)/127.5
            #images=np.squeeze(images)
            images=np.expand_dims(images, axis=0)
            disc_loss += train_discriminator(images)
            a1.append(disc_loss)

            #a1[i]=disc_loss
            #a2[i]=gen_loss
            i=i+1

            if disc_optimizer.iterations.numpy() % n_critic == 0:
                gen_loss += train_generator()
                a2.append(gen_loss)

and then plt.plot(a1) and a2.

For a learning rate of 0.0005 these are my losses

https://imgur.com/gallery/IlxnNre

(Not sure of best way to upload)

The generator loss really bothers me as do the images. The other learning rates don't seem to be much better. Even when I beefed up my network (the heaviest layers having 512 filters, these problems persisted)

How do I work this out as no matter what I do, my models refuse to converge

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.