6
$\begingroup$

I'm doing experiments with GAN. I've successfully trained GAN on 28x28px MNIST dataset (samples scaled to (-1,1) interval)

My next experiment is to train GAN on bigger images. My dataset consists of grayscale 128x128px images. I've added one more convolutional layer to both generator and discriminator and updated dense layers accordingly.

Sample from MNIST dataset and sample from my custom dataset: enter image description here

However, when I trained the model, all results from generator looks like this:

single sample

What am I doing wrong? Why does the generator generate only this noise pattern instead of samples from my dataset distribution?

Generator:

H = Sequential()

H.add(Dense(1024, input_shape=(noise_dim,)))
H.add(BatchNormalization())
H.add(Activation('relu'))

H.add(Dense(128*16*16))
H.add(BatchNormalization())
H.add(Activation('relu'))
H.add(Reshape([16, 16, 128]))

H.add(Conv2DTranspose(128, (3, 3), strides=(2, 2), padding='same'))
H.add(BatchNormalization())
H.add(Activation('relu'))

H.add(Conv2DTranspose(64, (3, 3), strides=(2, 2), padding='same'))
H.add(BatchNormalization())
H.add(Activation('relu'))

H.add(Conv2DTranspose(32, (3, 3), strides=(2, 2), padding='same'))
H.add(BatchNormalization())
H.add(Activation('relu'))

H.add(Conv2DTranspose(1, (3, 3), padding='same'))
H.add(Activation('tanh', name='out_image'))

gen_in = Input(shape=(noise_dim,), name='in_noise')
gen_out = H(gen_in)

Discriminator:

H = Sequential()

H.add(Conv2D(32, (3, 3), strides=(2, 2), padding='same', activation='relu', input_shape=(128, 128, 1)))
H.add(LeakyReLU(0.2))

H.add(Conv2D(64, (3, 3), strides=(2, 2), padding='same', activation='relu'))
H.add(BatchNormalization())
H.add(LeakyReLU(0.2))

H.add(Conv2D(128, (3, 3), strides=(2, 2), padding='same', activation='relu'))
H.add(BatchNormalization())
H.add(LeakyReLU(0.2))
H.add(Flatten())

H.add(Dense(1024))
H.add(BatchNormalization())
H.add(LeakyReLU(0.2))

H.add(Dense(1, activation='sigmoid', name='out_binary'))

disc_in = Input(shape=(128, 128, 1), name='in_image')
disc_out = H(disc_in)
$\endgroup$
1
$\begingroup$

Training GANs becomes much more difficult with increasing input dimensionality. I encountered that sometimes even small changes to your network can decide about convergence/divergence of your network (like initialization of your variables).

I recommend following the tipps from https://github.com/soumith/ganhacks.

Another thing I personally recognized is that you should not use as many Conv2DTranspose layers as you do in your current network. Try instead increasing the latent space.

$\endgroup$
1
$\begingroup$

The deconv layers are probably to blame. Check out this distill article for a fairly in depth discussion about how deconv layers create checkerboard artifacts. The gist is that deconv striding creates interference patterns which can cancel out if you're careful, but are more likely to be worsened as you add more deconv layers. The authors suggest using nearest neighbors for upsampling followed by a convolutional layer, which they call resize-convolution upsampling.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.