I am currently trying to train a GAN (based on proGAN) to produce images in a Vaporwave-style which is quite distinct.

The results so far have been underwhelming, which I suspect might be due to a very small training sample (only 909 images).

However in trying to optimize the model and my code I stumbled upon the following problem:

The output of the generator model has a different scale than my training samples!

My input data are image arrays scaled from 0-1, however the generator produces arrays scaled from -1 to 2!

Why does this happen? And is it a problem at all?

My code

You can find my code and the output so far here:

Vaporgan - Kaggle Notebook

  • $\begingroup$ The link to your code seems to be broken; it says "404. We can't find that page." $\endgroup$
    – noe
    Oct 30 '19 at 16:46
  • $\begingroup$ @ncasas You are right, the notebook was mistakenly on private. I changed it to public. $\endgroup$
    – Fnguyen
    Nov 1 '19 at 9:04
  • $\begingroup$ If you want to have a specific range of values at the output, you can modify the generator's final activation. For the output to be in $[0, 1]$ you could have a Sigmoid as final activation function. $\endgroup$
    – noe
    Nov 6 '19 at 8:32
  • $\begingroup$ @ncasas Cool thanks for the tip. If you want to type up an answer with code I'll happily upvote it. But I will try this anyway and report back if it works. $\endgroup$
    – Fnguyen
    Nov 6 '19 at 8:39

If the last layer of your output has no activation (e.g. a linear layer, a convolution), then its range is unbounded. Supervised training may lead the network to learn the appropriate output range. This does not seem to be the case with your GAN.

When you need the output of the generator to be constrained to a specific range of value, the safest approach is to simply force it by having the last layer use an activation function that does so by construction. As you need the output to be in $[0, 1]$, an appropriate activation function for the last layer of the generator would be a Sigmod.

  • $\begingroup$ This is very interesting, so the natural idea would be to let the model learn that [0,1] is the right output range. Using a sigmoid activation would be simply giving the model a nudge that might help it learn faster, correct? $\endgroup$
    – Fnguyen
    Nov 6 '19 at 10:00
  • $\begingroup$ Maybe not faster, but it will enforce by construction that the output is in the expected range, regardless of whether the model actually learned it or not. $\endgroup$
    – noe
    Nov 6 '19 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.