# GAN to generate a custom image does not work

I have been training a GAN in the cloud for some time now. I use Google's free credit. My laptop with a CPU doesn't seem to be up to the task.

The image I want to generate is this.

Even though the number of epochs is about 15000 I don't get anything close to the original.

This is the main code. I don't claim to fully understand the deep layers. It took a few days to even write this code. The rest of the code is boilerplate to train.

There is no compilation error and I look at the images using TensorBoard.

The output from the generator is (1024,1024) images. Should this be the same as my original which is a (299,299) images.

Should I calculate using formulas how each layer transforms the image to understand it better ?

How do I fix this ? I have mixed and matched API's just to create a working example assuming that doesn't create any problem.

X = tf.placeholder(tf.float32, shape=[None, 299, 299, 1], name='X')

Z = tf.placeholder(dtype=tf.float32,
shape=(None, 100),
name='Z')
is_training = tf.placeholder(dtype=tf.bool,name='is_training')

keep_prob = tf.placeholder(dtype=tf.float32, name='keep_prob')
keep_prob_value = 0.6

def generator(z,reuse=False, keep_prob=keep_prob_value,is_training=is_training):
with tf.variable_scope('generator',reuse=reuse):
linear = tf.layers.dense(z, 1024 * 8 * 8)
linear  = tf.contrib.layers.batch_norm(linear, is_training=is_training,decay=0.88)
conv = tf.reshape(linear, (-1, 128, 128, 8))
out = tf.layers.dropout(out, keep_prob)
out = tf.contrib.layers.batch_norm(out, is_training=is_training,decay=0.88)
out = tf.nn.leaky_relu(out)
out = tf.layers.dropout(out, keep_prob)
out = tf.contrib.layers.batch_norm(out, is_training=is_training,decay=0.88)
out = tf.layers.dropout(out, keep_prob)
out = tf.contrib.layers.batch_norm(out, is_training=is_training,decay=0.88)
print( out.get_shape())
out = tf.nn.leaky_relu(out)
tf.nn.tanh(out)
return out

def discriminator(x,reuse=False, keep_prob=keep_prob_value):
with tf.variable_scope('discriminator',reuse=reuse):
out = tf.layers.conv2d(x, filters=32, kernel_size=[3, 3], padding='SAME')
out = tf.layers.dropout(out, keep_prob)
out = tf.nn.leaky_relu(out)
out = tf.layers.max_pooling2d(out, pool_size=[2, 2],padding='SAME', strides=2)
out = tf.layers.conv2d(out, filters=64, kernel_size=[3, 3], padding='SAME')
out = tf.layers.dropout(out, keep_prob)
out = tf.nn.leaky_relu(out)
out = tf.layers.max_pooling2d(out, pool_size=[2, 2],padding='SAME', strides=2)
out = tf.layers.dense(out, units=256, activation=tf.nn.leaky_relu)
out = tf.layers.dense(out, units=1, activation=tf.nn.sigmoid)
return out

GeneratedImage = generator(Z)

DxL = discriminator(X)
DgL = discriminator(GeneratedImage, reuse=True)

• How many different images do you have in your training set? – JahKnows May 30 '18 at 15:30
• 134 images all of the same size. If that is really a problem them I need another dataset. Where ? – Mohan Radhakrishnan May 30 '18 at 15:33
• That is really insufficient to train a GAN sorry :(. However, luckily you can easily scrape the web for more random screenshots of pages. – JahKnows May 30 '18 at 15:33
• What are you actually trying to do? I am sure there is a better way you can do this. – JahKnows May 30 '18 at 15:34
• tens of thousands of instances for MNIST. For something as complex as your problem billions of instances. – JahKnows May 30 '18 at 15:50

Generating text as an image is extremely difficult and I have never seen a GAN applied in the image space to generate pages of text. The reason this is so hard is because of the way in which text is perceived by humans and the way a GAN works.

Humans read arbitrary symbols which are sequenced from left to right along the same line and combined into rows. Moreover, these symbols are combined into groups which represent words. This is extremely complex. The symbols must be intelligible, the words must be real ones as invented by humans. Lastly, the combination of words into sentences need to be logical and follow guidelines of human language. And even FURTHER the sequence of sentences must be coherent to transmit a message.

A GAN operating in image space will try to learn the distribution of the training set in a pixel-wise manner as that is your inputs. The distribution of the pixels will not effectively be able to group characters together in a logical manner, and the words will not be real, and the sentences will all be nonsense. You will most likely end up with blurry lines of random looking symbols, kind of like a zebra print.

Another problem is the amount of data you have. Even if this problem was possible with a GAN you would need tens of thousands of instances to train a GAN effectively.

# What I suggest

I suggest to read the texts that you are training with and use this data to train a LSTM which has been shown to be very effective for generating language. But, be warned even with the best LSTMs you rarely get text that can fool humans into thinking its real.

The LSTM will provide you with your text.

Then you can train a GAN to generate the characters of the alphabet and you can use this generator to print out the text that the LSTM generates.

• Could you point to some paper or material to understand why it is complex ? And I didn't get why you suggest the GAN should learn alphabets.I get LSTM's. – Mohan Radhakrishnan Jun 3 '18 at 14:43